Through the interaction between people and their surrounding environment, such as moving in a space, touching things, sitting on a chair, or sleeping on a bed, one can infer how the scene is set up and where objects are located. MIME creates a rich, imaginative, and three-dimensional environment by analyzing people's body movements. Many fields, including architecture, gaming, virtual reality, and synthetic data synthesis, can benefit from this technology. For example, AMASS has a large dataset of human 3D motions, but these datasets rarely include details of the three-dimensional environments in which people are situated.
Can they use AMASS to create credible three-dimensional scenes for all movements? If so, they can use AMASS to generate training data for real human-scene interactions. They have developed a new technology called MIME (Mining Interaction and Movement to infer 3D Environments) to address this problem. What makes it possible? The basic premise is that human movements in space define areas without furniture in the images. Additionally, this also determines the types and positions of three-dimensional objects when they come into contact with the scene; for example, a sitting person must be sitting on a chair, sofa, or bed.
Researchers from the Max Planck Institute for Intelligent Systems in Germany and Adobe have created MIME, a Transformer-based autoregressive 3D scene generation technology that transforms intuition into concrete form. Given an empty floor plan and a sequence of actions by a person, MIME can predict the furniture that will come into contact with the person. Furthermore, it can anticipate objects that do not come into contact with the person but interact with other objects and adhere to the spatial constraints imposed by the person's movements. They classify movements into contact and non-contact categories to provide conditions for creating three-dimensional scenes of human movements. They use POSA to estimate potential contact poses. Non-contact poses project the foot vertices onto the ground plane to establish the room's free space, which they record as a two-dimensional floor map.
Contact vertices predicted by POSA create three-dimensional bounding boxes that reflect contact poses and related 3D human models. Objects that meet the criteria for contact and free space are automatically used as input for the transformer. They extended the large-scale synthetic scene dataset 3D-FRONT and created a new dataset called 3D-FRONT HUMAN to train MIME. They automatically add people to 3D scenes, including non-contact people (a series of walking actions and standing people) and contact people (sitting, touching, and lying people). To do this, they use static contact poses scanned from RenderPeople and motion sequences from AMASS.
MIME creates a realistic 3D scene layout for input actions during inference. They select 3D models from the 3D-FRONT collection based on this arrangement; then, they fine-tune their 3D positions based on the geometric constraints between human positions and the scene. Their technique produces a collection of 3D objects that can support human touch and movement while placing real objects in free space, unlike pure 3D scene creation systems like ATISS. Their approach can develop objects that do not come into contact with people and can predict complete scenes rather than individual objects, in contrast to recent work like Pose2Room. They demonstrate that their method requires no adjustments on recorded real motion sequences, such as PROX-D.
Source: https://www.marktechpost.com/2023/06/20/researchers-from-max-plank-propose-mime-a-generative-ai-model-that-takes-3d-human-motion-capture-and-generates-plausible-3d-scenes-that-are-consistent-with-the-motion/
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.