After you get the handles and all of the screws out, You can pull out the bottom of the panel away from the door and then bang your hand up on the bottom of the panel. The whole thing slides up and away. Basically the top hooks over the lip of the door. It might help to squeeze the top of the panel into the door with your other hand at the same time as it will take some pressure off of the clips.
Let us know how it goes.