An Apple spokesperson told AllThingsD that users who wanted to access music from the cloud-based locker would still need to store the file on their device. What looked like streaming in the videos released yesterday was actually a 'simultaneous listen and download'. This reportedly applies to both music you've purchased from the iTunes Store and music that you've uploaded with iTunes Match.
Apple won't go into further detail about how the service will works - it was deliberately vague about it when it showed off the service at its developer conference last June, too - so we'll have to make some educated guesses here. My best hunch: If you don't "download" a music file to your library, it will sit in a more temporary cache, on a different part of your machine. Depending on the size of your machine's cache - it will presumably differ from, say, an iPhone to a MacBook - that file may occasionally be cleared out.