Naturally, you can work in a mixed reality environment with a connected Bluetooth keyboard and mouse, and you can put yourself in a focused position if you want to focus, or leave the vision mode on to make sure that your colleagues don’t see. taking pictures and giggling while wearing a silly headset to make things happen. It wasn’t clear whether you’d be able to connect the headset to a laptop for mixed reality, a feature found in the Apple Vision Pro.
Gemini in XR
Tapping the side of the headset brings up the app launcher, and this is where you can switch on Gemini if you want it to stay “on.” Once it’s on, there’s an icon at the top of the screen to let you know that everything you say and watch is being recorded by Gemini.
In pass-through mode, you can walk up to an object and ask Gemini about it—a Googler lowered the headset (before I tried) to someone wearing an FC Barcelona shirt and asked Gemini to find “the shape of this. group.” Gemini quickly registered the team’s name and pulled out search results with league standings and recent match results.
You can ask Gemini anything like this and it will respond with visual effects displayed on the headset. I asked it to “take me to Peru,” and it opened a 3D version of Google Maps. I was able to walk around and focus on Lima, and in cities where Maps already has many 3D models, you can explore the areas in more detail. You can keep talking to Gemini about this experience, so I asked questions like when would be the best time to visit and I got a quick answer.
In another example, I peeked inside a restaurant in New York City to tour this space. Google says it can use AI to put together photos of a place’s interior and display it to make it feel like you’re there. It did a great job, and I asked Gemini if the place takes reservations, without specifically saying the name, because I was staring at the name of the restaurant. It it does take a reservation, but Gemini couldn’t make one for me. (That integration may come later.)
Next, I watched a few videos on YouTube, where the 2D content looked sharp and colorful. Stereoscopic content was even better; my senses felt surrounded. I watched the hikers walking along the trail and asked Gemini where all this was, and he said, “New Zealand.” I couldn’t confirm that, but it looked like the right answer. I’ve watched the gappy playback of 2D videos as the virtual player adds depth and layering to make them feel 3D. I jumped into the Google TV app and enabled “cinema mode” to launch a virtual theater for watching movies and shows, just like on other VR headsets.
Circle to Search, a feature Google introduced earlier this year to Android phones, is also available on Android XR. Just walk to a visible object near you, press the top button on the headset, then press and draw a circle around the object you want to know more about. You will get a Google Search page with results.
Smart Glasses
Project Moohan feels like Google and Samsung are reaching the entire VR market, although the integration of Gemini gives their efforts a unique layer. However, I will admit that I was very excited to try the smart glasses, when Gemini felt like it could be very helpful. They did not disappoint. I went to another room and there were several pairs of glasses in front of me. Some were sunglasses, some with light lenses. Like headsets, you can get them loaded with your prescription. Google did not provide a name for the prototype glasses.
Source link