Sorry buddy, hate to break it to you but it's not obvious. I asked a bunch of people on this specific scene and there's no concensus. It's open enough for interpretation and some people saw it one way, other people saw it another. Unless you have a document from the writer saying that's how it was, it's not as 'subtle as Hulk beating Loki like a ragdoll' :rolleyes
My POV from the other camp is:
-Loki needs to operate the spear to brainwash people, there is no evidence he can also do this remotely
-Banner picks up the spear because HE is affected and the spear is the closest weapon to him; this doesn't prove that the rest of the room is affected because Loki clearly puts the spell on him when he's being escorted by SHIELD soldiers, not because the spear is in the same room as Banner
-The spear started glowing before Hawkeye arrives, implying that the glowing could simply be a homing beacon function, also evidenced when the spear is on display in Hawkeye's Quinjet HUD
-Lastly, the spear having an effect on the entire room tears down Whedon's attempt to show the tensions that already exist within the group; these people are supposed to not get along to begin with, without any magic involved
Not really wanting to get into a debate on this because like I said - it's not a plot driver. The main thing we as the audience need to understand to move the plot forward is that Banner is under Loki's influence, that is all.
