It looks like a spiralised Death Star on a tripod – but could it be the future of filmmaking? Lytro today unveiled what could be the next step in video technology, and they haven’t gone in the direction many industry observers expected. Lytro has already used its Light Field technology to build cameras that allow you to refocus a stills image after capture. They perhaps haven’t proved as popular as might be expected. Observers had expected them to make a video camera using the same technology, allowing you to shoot HD video and refocus (Light Field imaging not only captures information about colour and luminence values, but also the direction it is travelling in). Instead, the new Lytro Immerge camera uses it to construct a 360-degree virtual world. This is the closest that we’ve got to the Matrix.
The camera looks like a truncated sphere and is packed full of lenses and sensors to capture entire scenes and use this information to construct a computerised model of them. What previously required computer generation can now be captured as live action – in ‘cinematic quality’, the company says.
This from Lytro: “Lytro Immerge captures Light Field Volume – all the data from all directions at any location within a given volume – allowing virtual views to be generated from any point, facing any direction, with any field of view.”
In other words, it’s not just that you can look around from a fixed position; you can lean in or to one side as you would in real life. In the video here you can see how the camera allows you to ‘move’ in six degrees of freedom:
1 – move left or right
2 – look up or down
3 – move up or down
4 – look left or right
5 – move forward or backwards
6 – tilt head side to side
Kurt Akeley, the company’s CTO, explains more in this video: “We collect enough ray data that we’re able to recreate a model of the scene that’s being viewed. And that recreation is not only what colour are things in different directions, but actually how far away is each thing along each ray. As users move…we redo that computation at a kind of theoretical or conceptual level and that’s how we deliver an experience that’s truly immersive.
“We’re not just stitching together the images that are captured – instead, we are essentially rebuilding a version of the scene. As a result, we have very reliable depth information and that makes it possible, in subsequent processing, to merge computer generated imagery with the imagery which is captured.”
Along with the camera comes hardware, software and services so that content can be processed, edited and played back. It will be possible to play back on any of the leading VR devices – welcome given the plethora of headsets out there.
Clearly, the whole thing needs a lot to work – hence the fact it comes with its own server for all the storage and processing required. It isn’t going to work on your iPhone any time soon.
Citing CEO Jason Rosenthal, Mashable reports: “When it becomes available in early 2016 (Lytro is targeting January), the whole system will cost in the ‘hundreds of thousands’ of dollars…so it’s definitely something for studios and companies serious about creating VR content, not consumers.”
But look at how fast camera technology for movies has ended up in consumer hands. Cutting edge technology eventually ends up being highly portable as computer power and storage improve. Maybe in the future, we will all be wandering around with some version of this.
What might all this mean? As technology improves we will end up with something which in technical terms is visually indistinguishable from reality – and if Lytro Immerge lives up to the hype, this takes us a long way towards it. Lytro’s press release stresses the ability to seamlessly blend live action and computer graphics; just think how games could change.
If (note the caveat) it becomes commonplace, it may mean rethinking basic assumptions about factual and fictional filmmaking. Take news: If you feel like you’re in the middle of a natural disaster or a war zone, does that bring you closer to the event? The accusations we already see of poverty porn and voyeurism will be amplified a hundred times if you can “be” there without any of the danger.
Of course, there was a time when people assumed that everything would be shot in 3D and yet we aren’t donning those glasses to watch the news or the latest arthouse release. Another note of caution: Lytro haven’t to my knowledge shown this tech actually working as of yet so let’s see what it looks like when people actually get their hands on it.
You can read more about Lytro Immerge on their website.
This from Lytro:
Lytro Immerge: The World’s First Professional Light Field Solution for Cinematic VR
Delivers Lifelike Presence for Live Action VR through Six Degrees of Freedom
Provides Unparalleled Creative Freedom for Immersive Storytellers
Built from the Ground up to Blend Live Action and Computer Graphics
Configurable Capture and Playback to Support a Wide Range of VR Storytelling Needs
MOUNTAIN VIEW, Calif., (Nov. X, 2015) – Lytro Immerge is the world’s first professional Light Field powered solution to provide lifelike presence in VR with Six Degrees of Freedom (6DoF), addressing the challenges of live action immersive storytelling. Built from the ground up to seamlessly blend live action and computer graphics (CG), it features configurable capture and playback options. Watch a video about Lytro Immerge at https://lytro.com/immerge#video.
“Everybody is talking about Light Fields and nobody fully understands the potential yet. Light Field technology is probably going to be at the core of most narrative VR,” said Aaron Koblin, Cofounder and CTO of Vrse. “At Vrse, we’re just waiting for the moment when we have the tools. I think both the capture and playback of Light Fields will be the future of cinematic virtual reality.”
Lytro Immerge captures Light Field Volume – all the data from all directions at any location within a given volume – allowing virtual views to be generated from any point, facing any direction, with any field of view. This approach creates a breakthrough sense of presence and enables a 6DoF live action VR experience, previously only possible in CG VR experiences.
“VR is the next wave in cinematic content, and immersive storytellers have been seeking technology that allows them to fully realize their creative vision,” said Jason Rosenthal, CEO of Lytro. “We believe the power of Light Field will help VR creators deliver on the promise of this new medium. Lytro Immerge is an endtoend Light Field solution that will provide all the hardware, software and services required to capture, process, edit and playback professionalgrade cinematic content.”
Lytro Immerge is the first end-to-end VR hardware and software solution, built from the groundup for professional storytellers. The system is comprised of four main components:
Camera: Configurable dense Light Field camera array
Server: Storage and processing
Light Field Editor: Integrates with existing visual effects tools like Nuke
Player: Light Field video playback engine for leading VR headsets and platforms
When using Lytro Immerge, professional content creators will benefit from unique advantages and unprecedented capabilities:
Precise visual perspectives from any (supported) point of view
Seamless capture that requires no stitching
Capture once, playback on any device (optimized for highend headsets, but works with mobile)
Accurate horizontal and vertical parallax
Depth information in every frame
Supports any interocular baseline after capture
Highly configurable based on production (e.g., 360°, 180°, parallax, spherical, etc.)
Lytro Immerge sets a new bar for the production of cinematic VR content. It packs the results of eight years of Light Field research, a revolutionary new Light Field processing engine, modular data capture and processing architecture, and a custom capture rig to turn immersive cinematic storytelling with 6DoF into a reality. For more information on Lytro Immerge, visit www.lytro.com/immerge .
