I've devoted a lot of time so far to planning. However, there will be times things go off the rails. Some groups prefer more open ended games as well where the focus is on exploration instead of narrative. Some people just find it easier to improvise as they go along. Still, thinking on this topic I've come up with a few things that I hope will be helpful.
Wandering Monster Tables
These guys are very useful. I don't roll on them very much but they are a great source for inspiration. Think of them as representing what threats the local area contains and how frequent they are. If you don't have one, making a quick one before the session is easy and generally beneficial. That way, you know which creates are most likely in the area. You can also easily put something not on the table to mix things up.
Tactical Maps and Dungeon Tiles
It's much easier to use theatre of mind on the fly compared to using a map or dungeon tiles. Still, dungeon tiles and maps can be easily used with some practice. Maps can be simply thrown down on the table and traversed (might want some blank sheets to cover up where players haven't been yet). Dungeon tiles take a bit more time to set up and as such are harder to use on the fly, but you can think of quick designs, design tables for yourself, or use the tables from the Dungeon Master's Guide. Outside of rolling tables, dungeon tiles will require quicker thinking and also more set up time. With some practice, it can make for an enjoyable dungeon delve.
What Kind of Situations?
Dungeon delves are easiest for me to improvise on, especially when you already have a bunch of maps you could run through. More story focused things are harder, but with a bit of practice it can be done. I typically find that focusing on a few characters and having everything fall out from that is easiest for me and also gets the best results (easy to keep in mind at once). Also, don't forget the calm before the storm and mood when improvising. It's was very easy for me to skip on the details when I first started doing serious improvisation. Don't forget the signs, the sounds and other aspects of the environment that help contribute to the theme and feeling.
Not forgetting details is a big pitfall that can happen when improvising for the first time. However, this is also true for combat. The details are important for tactical considerations such as flanking, good archer positions (where archers can keep out of melee), hiding and many others. Maps, again, remedy this a little bit but not completely. Many times, you'll be forced to populate maps with objects as they are empty otherwise. Keeping a little pile of objects for this purpose helps remind me. Generally, it is easier to respond to what players do instead of throwing something brand new their way. As a result, if the players don't know where to go, it can be hard to think of something interesting to throw their way. As you improvise, keep a piece of paper to write down ideas that you can use later in such situations.
It's Not Easy
It's not easy, but it can be very rewarding to think on your feet. It also means you won't be railroading players since there is very little investment into preparation. Getting good at it also helps cut down on preparation time, which can be a plus if you are busy. It's also much better than missing sessions if you can do it well. Good luck to everyone running games this way and I'd love to hear of any other tips people are willing to share.