MIDI stands for Musical Instrument Digital Interface, a technical standard for digital communication between digital instruments and/or computers. It was invented sometime in the early 1980's with the advent of synthesizers & digital keyboards.
MIDI allows composers to program parts to be played back in real time, effectively compose in advance, just like writing a musical score.
MIDI can be entered and edited as musical notation, piano roll style notation, pure numbers, or captured from a live performance on a digital instrument. The end result is just a string of numbers specifying notes, timing, attack, volume, timbre, etc.. Just like musical notation, MIDI tracks can be played back with any sound generation device available (although the results may or may not be musical).
The MIDI playback stage can executed in any number of ways, a computer can control a physical electronic instrument, such as a digital piano (acting like a player piano), or it can control the playback of an electronic synthesizer which can be programmed to emulate real instruments or play purely electronic sounds.
A third playback method, the most widely used these days involves using MIDI playing back samples. Samples are live recorded snippets of real instruments being played, capturing their character over their musical range and various articulations, usually note by note. Most digital instruments these days use this method because of its realism.
GarageBand, and most other Digital Audio Workstations (DAW), contains both synthesizers and libraries of samples. Just like the old days of multi-track tape machines, multiple tracks can be assembled and overlaid from various sources to create the final product, including MIDI realizations, and live recorded instruments/vocals.
When all the tracks are played back in GB, the result is a mix of the MIDI tracks being played back along with any live recorded tracks. The output of this performance can then
be captured in a digital recording file (WAV, MP3, AIFF). Most of the music on this site is a combination of MIDI realizations along with live instruments and vocals.
Note: MIDI differs from digital recording/playback (WAV, MP3, AIFF), which involves a numeric representation of the actual sound waves, sampled at a very high frequency.