More Gemini Features Coming to Google TV
Google TV integrates Gemini AI for image/video generation, enhances Google Photos, and adds a dedicated YouTube Shorts feed to the homepage.
Read moreBelow is a guest article by researchers from the Max Planck Institute for Intelligent Systems. In this article, the researchers describe a new method for dense 4D reconstruction from images or sparse point clouds.
Intelligent agents that interact and navigate in our world must be able to reason in 3D. Therefore, there has been a lot of interest in learning-based 3D reconstruction in recent years. Such techniques use neural networks to process some input information, such as images or sparse point clouds, and output the 3D geometry of individual objects or scenes.
One particular challenge in 3D reconstruction is the representation of the geometry we use as output. On the one hand, this representation should have enough expressiveness to allow us to represent all the complex geometries that occur in the real world. On the other hand, it should work well with modern machine learning techniques, especially deep learning.
Recently, our team proposed a new learning-based 3D reconstruction output representation called "occupancy networks," where the geometry is represented by a deep neural network that distinguishes between the interior and exterior of objects. We found that this representation is both powerful enough to represent complex geometries and very suitable for the paradigm of deep learning. It can be parallelized effectively not only on object instances but also on query points, allowing us to take full advantage of the latest NVIDIA GPUs.
For example, this allows us to train a neural network that can reconstruct humans from sparse 3D point clouds.
We use the RTX NVIDIA GPU with cuDNN-accelerated PyTorch deep learning framework for training.
While this works well, it is not entirely realistic: in the real world, objects are in motion, so humans do not remain still but move. Therefore, our input would look like this:
Of course, we could treat each time step separately and apply it to the network. However, this would be very slow (after all, we cannot reuse information from previous time steps), and even worse, we would have no correspondence between the different time steps. Where does the fingertip end up 2 seconds later? We don't know because we cannot identify the specific point on the body (e.g., fingertip) in later time steps. Can we do something smarter than this?
Our main insight in this project is that we can separate the representation of the shape and motion of 3D entities ("unentangle" them). We not only use occupancy networks but also employ two networks: an occupancy network and a velocity network. The occupancy network represents the shape of the 3D object, while the velocity network defines the motion field that changes over time in 3D space. We can visualize it as follows:
In the inference process, we can use the occupancy network (as in the previous paper) to extract a mesh at time 0 and then propagate the vertices forward in time using the velocity network by solving an ordinary differential equation.
In the training process, we sample random points in 3D and return them in a timely manner to determine their positions at time 0. We then evaluate the occupancy network at these assumed positions and compare it with the ground truth occupancy at the sampled times. This way, we do not even need correspondences between different time steps in the training data! However, if they are given, we can easily merge them by comparing the positions of the points with the positions they should go to according to the vector field they follow.
In summary, the key idea is that we disentangle the shape and motion of 3D objects. When we represent motion with a continuous velocity network, we automatically obtain correspondences between points in space across time steps. This also makes the inference step much faster!
Let's revisit our example from the introduction. How does the "occupancy flow" perform? The results are as follows:
We can see that the "occupancy flow" reconstructs the 3D motion in a reasonable way and correctly shows the correspondences (represented by colors).
In our paper, we conducted various other experiments. For example, we used a generative model to transfer motion from one shape to another:
Google TV integrates Gemini AI for image/video generation, enhances Google Photos, and adds a dedicated YouTube Shorts feed to the homepage.
Read moreAlphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Read moreAnthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Read moreApple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Read moreGoogle launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
Read moreYouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Read moreGoogle grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Read moreMistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.
Read moreCustom subliminal audio maker

AI Social Media Manager

AI Search Visibility

Workout and Calorie Tracker
Stock research platform

AI interview assistant

Scientific figure generator

AI courses with real code