PhD 3D image sensor researcher Daniel Lau explains the difference between Kinect 1.0 and the Xbox One’s Kinect 2.0, and how the 3D sensing tech in the devices work. …
One of the big selling points of the Xbox One is that you can talk to it – but you don't have to. Are you using the console's voice commands?
The Xbox 360 and its Kinect were capable of receiving voice commands, but it was borderline useless. I made an effort to use it on a few occasions, but it never worked without repeated attempts and the raising of my voice. I gave up and started using the controller every time.
The Xbox One's Kinect works significantly better. It's not quite Start Trek technology where I can talk to the console using my normal speaking voice and it works every time, but I have been very impressed with its consistency and ability to recognize my voice.
I am still using the controller to select applications and for the majority of functions, but I have been using the voice commands to pause and play movies and to turn on the console. Those are functions I want to use, but the controller is usually off.
What about you? Are you using the voice commands? Or are you sticking with the controller?
The list documents everything from basics like “Xbox, on,” to more specific commands regarding the console’s Snap functionality. It also reveals a handful of alternate commands, allowing you to amaze and bewilder your friends by using the more colloquial “Xbox, show my stuff!” instead of “Xbox, go home.”
Apple of iProduct fame has acquired PrimeSense, the Israeli firm behind the 3D sensing tech in the original Kinect for Xbox 360. No official price tag was put on the purchase, but rumors of the acquisition last week speculate a cost of $ 345 million.
“Apple buys smaller technology companies from time to time, and we generally do not discuss our purpose or plans,” said Apple to the BBC.
PrimeSense noted the acquisition to Engadget in a brief statement, “We can confirm the deal with Apple. Further than that, we cannot comment at this stage.”
The fateful day of the Xbox One and the new Kinect is upon us, and rather than spend the day playing new video games, Dan Ryckert and myself decided to grief and haze the console and the new Kinect camera to see how much we could confuse it.
Standing the console up
Microsoft does not recommend standing the console up like the majority of owners did with their Xbox 360s. When I stood the console up, it not blow up or immediately erupt into flames. It appeared to continue running with no problems, but it did not recognize the disc I placed in the console. It took the disc in and ejected it without issue while standing up, but it would not play, recognize, or install the game.
Distinguishing between cartoons, mannequins, and real people
For some reason, we have a mannequin in the offices of Game Informer. I don’t know why it’s here, but it’s been here since I started, and I hope to never find out where it came from so it can retain its mystery. The Kinect recognized it as a person, immediately and consistently marking it as a guest on the console. This is understandable as the mannequin has the same proportions as a human.
Standing behind the mannequin, I placed my arms in the air, and the Kinect thought that he sprouted human arms. I was able to navigate the menus while standing behind the mannequin.
For the next test, we decided to draw pictures of a body and a face and see if the Kinect was able to distinguish between them and real humans. The drawing of the person (which admittedly lacks detail) was never picked up by the Kinect.
The drawing of the face however, was picked up and registered as a guest on one occasion, but it wasn’t consistent. The Kinect realized that it was looking at a drawing of a face and a human, and nothing more.
Similar to the mannequin, we also have a mysterious giraffe statue in our office. Again, I don’t know why it's here, I don’t know where it came from, and I don’t intend to find out. For the most part, the Kinect understood that the giraffe was not a human being, but near the end of our experiments, it, strangely, started to log in the giraffe as me. Even while standing next to it, it still registered the giraffe as me, and me as a guest. It only happened on the one occasion, though.
As part of the promotional material for Batman: Arkham Origins, we were sent plastic Black Mask masks. On every occasion, once I was wearing the mask, it did not recognize me at all. If I walked out of the range of the camera and took off the mask and re-entered the frame, it would appropriately log me in as Kyle. We also tried taking the mask off while on camera, and while it signed me back in after removing the mask twice, it never worked for Dan.
Placing the Kinect upside down
Once the Kinect was turned upside down, it was useless in recognizing us. It couldn't log in or register anybody.
Activating Xbox commands over Skype and intentionally confusing the microphone
After downloading Skype, I called Dan who was at his desk in another section of the office and turned the volume up on the TV as loud as I could without disturbing others. Dan shouted commands, attempting to open Internet Explorer and Snap in other applications by saying, “Xbox, etc.” over the Skype conversation. It never worked, but I was still able to use voice commands. The device is skilled at distinguishing voices emanating from the television, and voices emanating from in front of the TV.
We also tried to confuse Kinect by standing on opposite sides of the room and shouting commands going back and forth between single words in a sentence. For example, one of us said, “Xbox,” and then the other said, “Bing,” and the other said, “Game,” and the other said, “Informer.” It confused the Kinect somewhat, but the majority of our attempts resulted in return searches of the words we were searching for.
We successfully tricked the Kinect on many occasions, but we were using the device in ways not originally intended. It did surprise us though, like when it understood that I had removed my mask and when it was able to easily distinguish between the voice from Skype and the voice from the person in the room. It’s not perfect, but when compared to the first Kinect, Kinect 2.0 is a vast improvement.
Just hanging out with my "guest" on Xbox One. https://t.co/iZ8vpKi2Ey
— Kyle Hilliard (@KyleMHilliard) November 22, 2013
“The key to replicating sensory experiences is turning actions into symbols,” states Hidetaka “Swery” Suehiro, director of the upcoming Xbox One Kinect-controlled D4, in this GDC Next 10 special session. …
Sure, a lot of games have 3D graphics, with fully rounded character models and everything, but their playgrounds are essentially flat – Avatar in 2D. The Secret Castle is reach-through-the-screen 3D (without the glasses), letting players manipulate the setting to see around objects, new solutions popping out from behind pillars and stashed under toys.
The Secret Castle launched on iOS from Platronic Games earlier this year, using motion controls that allowed players to tilt their screens and peer behind things in the game’s environment. It’s free to try, like a lot of games in the App Store – and that last part is one of its problems.
“The game has sold a couple thousand copies since launch,” Platronic Games founder John Francis tells Joystiq. “The reason I’d say that was poor is because even with a modest budget we never really recouped all sales. Everyone on the team was very passionate about the game and working for free on their nights and weekends.
“Also, our sales model was largely traditional in a growing free-to-play and socially media-driven market. Without Facebook plugins or a pre-established hardcore fanbase like you might find with platformers or tactics games, telling people you reinvented the hidden-object genre was a tough sell.”
Platronic wants to give The Secret Castle one more shot at success, this time with a different kind of motion control: Kinect.
Kinect is an impressive piece of technology, but it isn't intelligent. If you say or do something it hasn't been programmed to recognize, you won't get your desired result. How do you know what Kinect understands? With these two handy cheat sheets.
The images below outline the specific voice commands and motions that Kinect responds to. Save them and print them out for easy reference if you want to avoid prolonged bouts of random shouting and gesticulating.
Apple is currently in talks to purchase PrimeSense, the original suppliers of Microsoft’s Kinect motion-sensing technology, according to All Things D. The acquisition was first reported by Isreali publication Calcalist, which indicated that the purchase will amount to $ 345 million.
Founded in 2005, PrimeSense contributed its 3D sensing technology to the Xbox 360 version of Kinect. Microsoft since went with in-house developers for the latest iteration of the Kinect for Xbox One, the company’s latest console set to launch this Friday. Apple may be allegedly targeting PrimeSense for the company’s Capri technology, a newer form of its 3D sensing tech used in mobile devices such as Google and Samsung’s Nexus 10 tablets.
In the early days of the marketing for Xbox One, many gamers were turned off by the publisher’s insistence on showing off UI and entertainment features more than actual games. Months passed and Microsoft tuned down the multimedia hype, focusing instead on satisfying gamers after a disastrous E3. With many of these restrictive features (online check-ins, no used games, etc) now a thing of the past, it’s a good time to revisit the unique UI of the upcoming console. I had a chance to see it in action at a recent Microsoft event, and now have a better understanding of why the publisher was pushing these features so hard early on.
At first glance, the Xbox One dashboard isn’t strikingly different than the 360’s. It’s presented with the same Windows 8-style colored boxes as the previous console, and many features make their return. The main screen allows players to jump into their game and access several recently-viewed apps. Moving one screen to the left accesses your favorites via the pins menu. To the right of the main screen is the store section, which features tabs for games, movies, television, and music. Overall, the core sections of the new dashboard seem more streamlined and simple than those on the 360.
Microsoft’s initial plan to require Kinect makes more sense now that I’ve seen just how integrated it is in the UI. Logging in no longer requires going through menus and selecting user profiles with the controller. Kinect can now recognize up to six individuals in a room, and will automatically log in each user. Facial recognition was supposed to be a function of the original Kinect, but it rarely worked. Microsoft claims that the new tech in Xbox One’s Kinect will be much more reliable in this regard.
If multiple people (up to six) are in the same room, Kinect will recognize them and automatically sign them in without the use of controllers or menus. Let’s say Person A is the primary user, and the menus are populated with their pins and theme selections. All Person B (or C,D, etc.) has to do is say “Xbox, show my stuff.” The theme and pins change accordingly, as do any favorite cable channels in the TV section.