A Full Vision Pipeline: Using MIPI in Agilex® 5 and Agilex® 3 FPGAs - A Full Vision Pipeline: Using MIPI in Agilex® 5 and Agilex® 3 FPGAs MIPI (Mobile Industry Processor Interface) has become one of the most common ways to connect image sensors, cameras, and displays in compact embedded systems. For designers building industrial vision, robotics, medical imaging, smart cameras, edge AI, broadcast, or display applications, MIPI connectivity is often the first step in the video pipeline, but it is rarely the last. Agilex® 5 and Agilex® 3 FPGAs now give Altera customers a direct path to bring MIPI camera and display data into programmable logic. With native MIPI D-PHY connectivity for CSI (Camera Serial Interface) and DSI (Display Serial Interface) applications, designers can connect sensors and displays more easily while using FPGA fabric for real-time processing, aggregation, adaptation, and control. Agilex 5 and Agilex 3 MIPI D-PHY support at a glance Capability Agilex 5 Agilex 3 Maximum MIPI D-PHY lane rate Up to 3.5 Gbps per lane Up to 2.5 Gbps per lane Supported lanes per interface 1, 2, 4, or 8 lanes 1, 2, 4, or 8 lanes Maximum interfaces per device Up to 28 Up to 14 Maximum raw bandwidth per interface Up to 28 Gbps Up to 20 Gbps Maximum raw bandwidth per device Up to 784 Gbps Up to 280 Gbps Customer value High-bandwidth sensor/display links, multi-camera ingest, aggregation, adaptation, and direct FPGA processing Cost-optimized MIPI connectivity for embedded vision and display designs Note: Maximum supported rate, lane count, interface count, and bank usage vary by device, device group, package, HSIO bank, reference condition, and design configuration. Consult the current product documentation for supported configurations. For real-world context, a 4-lane Agilex 3 MIPI D-PHY interface at 2.5 Gbps per lane provides up to 10 Gbps of raw bandwidth. That is enough raw capacity for a RAW10 4K image stream at roughly 100 frames per second, or an 8K RAW10 stream at roughly 25 frames per second, depending on image format and system overhead. A 4-lane Agilex 5 interface at 3.5 Gbps per lane increases the raw interface bandwidth to 14 Gbps, creating headroom for approximately 140 frames per second at 4K RAW10 or around 40 frames per second at 8K RAW10. The exact frame rate depends on resolution convention, blanking, packet overhead, metadata, and margin, but the example shows why aggregate MIPI bandwidth matters for high-resolution sensors and physical AI systems. Why aggregate MIPI bandwidth matters MIPI D-PHY bandwidth is often described per lane, but system capability is determined by the total bandwidth available across the interface and the design. In an eight-lane Agilex 5 configuration, customers can use up to 28 Gbps of raw D-PHY bandwidth for camera or display pipelines. That headroom matters because real image and display systems carry more than active pixels. Designs also need room for higher bit depth, HDR or multi-exposure data, embedded metadata, protocol overhead, blanking intervals, and margin for future sensor or display upgrades. For customers, aggregate capacity can support higher-resolution sensors, higher frame rates, multiple streams through virtual channels, and more room for feature growth while staying within a compact MIPI connection. Even incremental per-lane improvements become more valuable when multiplied across several lanes and across multiple MIPI links in the system. Scaling MIPI across multiple point-to-point interfaces High-bandwidth vision and display systems often need more than one MIPI connection. Physical AI systems, for example, are driving the need for more cameras, image sensors, and real-time data sources at the edge. Robots, industrial machines, medical imaging systems, smart cameras, and autonomous platforms increasingly need to capture multiple views of the physical world, preprocess sensor data close to the source, and deliver that data efficiently to downstream processors. Agilex devices can scale MIPI D-PHY connectivity within a single interface and across multiple point-to-point interfaces, giving designers more ways to balance sensor count, lane count, pin usage, and bandwidth. Supported HSIO banks can be configured for multiple MIPI D-PHY interfaces, giving designers flexibility to connect several sensors directly to the FPGA. Once the data enters the FPGA, designers can preprocess, synchronize, filter, format, or aggregate sensor streams before passing them deeper into the system. The aggregated output can move through a faster, wider MIPI interface or through high-speed links such as PCIe, USB, or Ethernet, which are all available as hard Ips IPs in the FPGA. This system-level scaling is important because MIPI bandwidth compounds across every lane and every interface. A high-capacity single interface helps with demanding sensors or displays, while multiple point-to-point interfaces help system architects connect several MIPI devices directly to FPGA logic for aggregation, preprocessing, format conversion, or transport to another interface. What customers can build with MIPI plus FPGA fabric Once MIPI data enters the FPGA, designers can add custom image preprocessing, multi-sensor aggregation, format conversion, display bridging, low-latency control, compression, AI preprocessing, or video transport functions close to the sensor or display. This flexibility is useful when systems need to support new sensors, adapt to processor interface limits, or differentiate through custom video and vision pipelines. Part of a broader video and vision ecosystem MIPI D-PHY support in Agilex 5 and Agilex 3 is part of the broader Altera Video Solutions Stack, which brings together connectivity IP, image and video processing IP, reference designs, development kits, documentation, and partner solutions to help customers move faster from evaluation to production. Example: MIPI as the Front Door to an AI Vision Pipeline The Holoscan Sensor Bridge MIPI to 10GbE/25GbE reference design for Agilex 5 shows how MIPI connectivity can become part of a complete AI vision workflow. In this design, image data enters the FPGA through MIPI D-PHY and MIPI CSI-2, then moves into FPGA fabric where the CSI-2 pixel stream is converted to AXI4-Stream for connection to additional video and vision processing blocks. From there, the design connects to NVIDIA Holoscan Sensor Bridge IP and Altera Low Latency Ethernet IP, creating a path from MIPI image sensors to Ethernet-based transport and Holoscan processing. This is a useful example of why native MIPI support matters. It helps designers move sensor data directly into a programmable pipeline, where the FPGA can receive, adapt, process, and transport video data before it reaches the AI processing environment. For customers building robotics, medical imaging, industrial vision, or edge AI systems, this type of architecture can reduce integration work and provide more flexibility than a fixed sensor-to-processor connection. MIPI brings the image data into the FPGA, and the broader video solution stack helps turn that input into a system-level vision pipeline. Built for implementation, not just connectivity The MIPI D-PHY IP user guide also provides implementation depth that may be useful to technical readers. The IP includes AXI-Lite register access, generated design examples, simulation support, traffic generation and checking, Signal Tap based validation, and documented pin planning, calibration, equalization, clocking, and reset guidance. This helps position MIPI support as part of a practical design flow rather than a standalone interface checkbox. Explore the Altera Video Solutions Stack to find reference designs, IP, development platforms, and partner solutions for your next FPGA-based video or vision design. Discover the 4K AI Multi-Sensor Camera Reference design Learn how the 4K Multi-Sensor HDR Camera Solution system works