Why Robot Fleets Need a Data Architecture
Key takeaways
Assuming each AMR retains 30 to 100 GB of compressed or selectively captured camera, LiDAR, and control data per shift, a ten-robot fleet produces 300 GB to 1 TB. At 100 robots, that rises to 3 to 10 TB per shift, pushing scheduled file copies and object-storage uploads beyond the spare capacity of many site networks.
Teams must decide which periods deserve full-fidelity capture, which can be reduced to lower-resolution telemetry, and which can expire locally.
The same data must support two different workloads. Operations teams need current fleet state and rapid incident retrieval. Training teams need synchronized, replayable episodes with stable schemas and consistent identifiers.
A scalable fleet data path keeps durable streams at the edge, assigns each robot its own identity and storage quota, promotes full-fidelity segments when defined events occur, and maintains one indexed timeline that incident investigators and training jobs can query independently.
Robot fleets once recorded data mainly for debugging. After an incident, an engineer copied a bag file from the machine, replayed it on a laptop, and deleted it once the issue was resolved. That process breaks down when the same recordings must support model training, remote operations, safety investigations, compliance reviews, and customer reporting across a fleet too large for engineers to inspect files by file.
Public robotics datasets show how much effort is required to produce reusable data. DROID contains 76,000 demonstration trajectories covering about 350 hours of interaction across 564 scenes and 86 tasks. Collection took 12 months and involved 13 institutions, 18 robot setups, and 50 human operators. Open X-Embodiment combined 60 datasets from 21 institutions into more than one million trajectories across 22 robot embodiments, 527 skills, and 160,266 tasks. These projects required coordinated hardware, operators, calibration, synchronization, metadata, and quality control. A warehouse running 50 AMRs across two eight-hour shifts generates about 800 robot-hours of operation each day. Most of that data is not preserved in a form that can be searched, replayed, or reused for training.
The first constraint is data volume. One 1080p camera recording uncompressed 8-bit RGB at 30 frames per second produces about 6.2 MB per frame, 186 MB per second, and 670 GB per hour. Encoding the stream with H.264 at 10 Mbps reduces it to approximately 4.5 GB per hour, although compression can remove image detail needed for some perception workloads. A 128-beam LiDAR producing 2,048 columns at 10 Hz generates about 2.6 million points per second. At four 32-bit fields per point, that equals roughly 40 MB per second or 150 GB per hour. Recording position, velocity, and effort for a 30-degree-of-freedom platform at 1 kHz adds about 360 KB per second, or 1.3 GB per hour, before timestamps and message overhead.
Actual fleet storage depends on compression, sampling rates, sensor count, and capture policy. Assuming each AMR retains 60 GB of selected camera, LiDAR, and control data per shift, 40 robots generate 2.4 TB over eight hours. Uploading that volume during the shift requires approximately 83 MB per second, or 670 Mbps, before protocol overhead, retransmissions, and traffic bursts. Even a site with a gigabit-class connection may struggle to dedicate most of its capacity to robot uploads while supporting fleet management, remote-operator video, business systems, and other plant workloads. Docking and shift changes can concentrate uploads further. Once the fleet reaches this scale, recording becomes a selection problem.
The next constraint is local storage. A nominal 2 TB drive holds about 33 shifts at 60 GB per shift, equal to roughly 16 days of operation at two shifts per day. Doubling the logging rate reduces that buffer to about eight days. Previously retained files, drive overhead, and interrupted uploads reduce it further. When the disk fills, the recorder either stops writing or deletes older data. Both outcomes can remove the information engineers need. A verbose debug stream left enabled after testing can consume the space reserved for the minutes around a safety event.
Selection depends on metadata recorded at ingest. Each recording or message envelope needs persistent associations with the robot, site, software and firmware versions, mission identifier, timestamps, and capture trigger. The fleet can then apply conditional replication. Safety stops, operator takeovers, protective faults, failed grasps, and low-confidence perception events can be transferred immediately at full fidelity. Routine navigation from an uneventful shift can remain local until its retention window expires. This directs limited bandwidth toward the data with the highest operational or training value. Without reliable metadata, teams are left with broad policies such as uploading everything or retaining a fixed sample.
A scalable fleet data path can use the same principles as the messaging system around it. Streams can be addressed hierarchically by site, robot, and sensor, allowing a consumer to subscribe to one robot’s fault events or every relevant stream at a facility. Each robot can carry its own identity, permissions, and storage quota, limiting the data a compromised or misconfigured unit can publish. Durable storage at the robot or site gateway allows recording to continue during an uplink failure and forwards selected data when connectivity returns.
Each consumer tracks its own place in the stream, allowing incident reviews, operational dashboards, training jobs, safety analysis, and compliance exports to read the same underlying data independently and at their own pace. Event triggers can promote full-fidelity time windows, while a shared index over timestamps, identities, missions, software versions, and labels gives engineers and training systems a consistent view of the fleet.
About 4.66 million industrial robots were in operational use worldwide in 2024, while major open manipulation datasets contain data on the order of one million trajectories. The comparison does not mean every deployed robot can immediately produce usable training data. Most operating hours lack the synchronized observations, actions, calibration, labels, and permissions required for reuse. The opportunity lies in building a recording path that preserves selected data in a form that incident investigators, fleet operators, and training systems can all query. That architecture becomes harder to retrofit with every robot added to the fleet.
Google announced Gemini Robotics 2, the latest version of its robotics foundation model designed to improve robot reasoning, motion planning, and task execution. The update introduces separate models for high-level task planning and low-level robot control, enabling robots to interpret instructions, reason through individual movements, and execute complex manipulation tasks with greater precision.
Intel announced new investments in OpenVINO Physical AI, extending its open-source framework to robotics and industrial automation. The company reported that more than 130 customers are adopting or evaluating Intel edge processors for robotics and Physical AI applications. Intel also highlighted collaborations with Foxconn, Siemens, and Hitachi to support deployment across vision, reasoning, and motion-control workloads.
Agency Tool Company, founded by former Scythe Robotics leaders Jack Morrison and Davis Foster, has launched ATC Deploy to simplify software updates across robots operating on unreliable or low-bandwidth connections. The platform transfers only changed data, resumes interrupted uploads, supports Docker and OCI images, operating-system images, and file trees, and lets teams define how software restarts after deployment. The company says updates can move up to 20 times faster than Docker pull or SCP and is testing the product with Burro, Gather AI, and Tempo Works.
MulticoreWare announced an expanded collaboration with AMD to accelerate autonomous robotics on AMD Ryzen AI platforms. The demonstration at AMD Advancing AI 2026 showcased Vision Language Action models running in real time on embedded hardware, enabling robots to combine visual perception, language understanding, and physical actions within a unified software stack. The collaboration focuses on deploying multimodal AI at the edge, allowing autonomous robots to execute complex tasks with lower latency and improved energy efficiency while reducing dependence on cloud inference.
SINTRONES announced its latest rugged Edge AI computing platforms for manufacturing, robotics, and industrial vehicles ahead of Automation Expo 2026. The systems are designed to run AI vision, robotic perception, and autonomous decision-making directly at the edge, supporting continuous operation in harsh industrial environments. The company highlighted real-time inspection, robotic automation, and machine vision as key deployment scenarios.
Personal.ai builds an individual AI model for each user, trained on the memories that person creates through conversations, voice recordings, and online activity. Every user generates several parallel data streams at once, and each message carries unique content that cannot be regenerated if lost. The platform runs thousands of these streams simultaneously, holds the data until a worker is free to process it, and routes results into separate storage systems for training and retrieval.
The first version of the architecture ran on AWS Lambda functions with SQS queues and a Redis cache for persistence. A bug in the health checks produced a massive AWS bill, bottlenecks appeared in the data pipelines, and the expensive GPU instances used for model training were difficult to manage precisely. The team moved the backend onto Kubernetes, with each user and their streams running as an isolated micro-pod, and went looking for a messaging layer to match. Kafka was ruled out for its heavy infrastructure and management overhead, while RabbitMQ and MQTT could not easily run active-active configurations at scale or carry streaming data without significant resource cost.
NATS with JetStream replaced the queuing service, the streaming pipeline, and the standalone Redis cache with a single system. Audio, text, and activity data are published to per-user subjects, persist in durable streams until processing completes, and land in the databases only after the work is confirmed. The built-in key-value store took over caching and materialized views, which removed an entire component from the architecture. Because the platform mixes many languages and external services, including automated speech recognition, the breadth of available NATS clients let the team connect and monitor everything over one fabric.
Running on Synadia Cloud removed the operational load of maintaining the messaging infrastructure itself, and the team reports spending less than one full-time engineer on managing the entire layer. Leaf nodes extend the same fabric toward edge devices, which opens a path to running capture and processing directly on smartphones while cached data persists until cloud GPUs finish their work.
Personal.ai once needed a queuing service for messaging, a streaming pipeline for user data, a Redis cache for persistence, and separate plumbing to reach edge devices. Each system added its own failure modes and scaling behavior. Collapsing them into one messaging fabric meant that irreplaceable user data now moves, persists, and recovers through a single system, and the team can reason about the whole data path in one place.
Unlocking the Power of Time Series Databases for Industrial and Robotic Systems (The Robot Report Podcast)
Doug Pagnutti of Tiger Data explains how time-series databases manage the continuous sensor, controller, and telemetry data produced by industrial and robotic systems. He covers how TimescaleDB extends PostgreSQL with automatic partitioning, compression, and continuous aggregates, allowing teams to query live and historical data without maintaining a separate database. The discussion also examines buffering data during connectivity outages, operating across edge and cloud environments, and combining telemetry with spatial and operational metadata for robotics and AI applications.