WarmGlass.com

I have done about 10 firings on my kiln and have not always needed to put kiln wash on the shelf each time. My last firing which was a bottle slumping put some indentations in the kiln wash when the bottles were removed and made it uneven. I tried to clean all of the wash off the kiln shelf and redo it but had a hard time removing it so I decided to flip the shelf over and use the other side.

So I don't build up a "kiln wash crust" again, I am wondering if I am supposed to completely clean the kiln wash off after each firing and reapply?

You can get quite a few loads done with one application of kiln wash. When it starts chipping off it is time to remove it. I use a hand sander with rough sand paper until it is completely gone. The shelf just gets smoother and smoother. Be sure to wear a mask. It can be a dusty job.
When doing something extra hot like pot melts I sand it down with each firing and reapply fresh.

drywall sanding mesh

The hotter you fire, the less durable is the kiln wash. Keep your firings under 1400ºF, and it will last a very long time. I advise at least a thin fresh coat after you have fired over 1400. Many times you can get away without this, but sometimes a white scum is left. I hate it when that happens. As others said, when you fire hot, you want a good fresh coating. After it has chipped, the only way to get rid of the ridge is to remove it all. I like Kevin's open mesh sanding screen as the most effective abrasive that doesn't get clogged up.

I use a bit different strategy. For firings above 1400, I make my own kilnwash mixing about 80% finely ground alumina hydrate with 20% EPK, by weight. This mixture is pretty easy to clean off each time. I work primarily with float glass, which requires a thinner coating than the fusing glasses. When I work on hard shelves, I usually paint a pattern on with my kilnwash which leaves a decorative imprint. For most of my flat shelf work, I fire on fiber board that has been dusted with alumina hydrate, and no wet kiln wash. Much of my work is done on the kiln floor with either a sandbed or powdered fiber blanket. These are the easiest approaches for me, in my kilns.