APUs are like small jet engines, small enough to be started electrically. So you would use the power of the battery or an electric ground power cart to spin up the APU by its electric starter motor, and then introduce fuel into it once its spinning fast enough, just like a jet engine is started. Depending on the design, the electric motor will then either act as a generator to supply electrical power back to the aircraft or decouple when a separate generator is used.
The APU will spin up, and besides spinning the electric generator, some pressurized air will be 'stolen' from its compressor to supply "bleed air" for engine starting, the air conditioning packs, etc...
Big APUs like on the Boeing 777 have actually two methods of starting: They have both an air-driven and an electric driven starter motor. The method of operation is as follows: When the APU is started with the engines off, electrical power from battery or ground connection is used. After the flight, the APU will be restarted before the main engines are turned off. In that case, the APU will be started by the air-driven motor rather than the electric motor, with pressurized air supplied by the still-running main engines.