"Assistant" does not work only based off a user's voice inputs, but Apple will take information securely from user's devices like location, contact's information, and music metadata to formulate the best information to serve a user's request. Since Apple will be taking information such as phone contacts and location, we truly think that "Assistant" will work just like the Siri iPhone application does; but fully integrated into the operating system. We can imagine a user asking their iPhone "Assistant" to setup a movie with one of their friends. The user might say "setup movie with Mark" and based on Mark's contact info and the user's location data, will be able to offer tickets to a local theater and send Mark the information.
9to5Mac believes this speech data will be crowd-sourced using an anonymous process similar to Apple's Genius functions. They also suggest that the Assistant will be able to interpret voice input and and speak back intelligently.
The thinking is that Apple might make this feature an iPhone 5 exclusive to make the new device more enticing.