FPGA passive serial configuration -- detect as PCIe device in Linux after configuration - FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi In passive serial configuration the Linux platform powers on and is booted with FPGA in unconfigured state -- our RBF file is then configured via passive serial and working but is not detected as a PCIe connected device in Linux -- the command "echo 1 > /sys/bus/pci/rescan" has no effect... Is there a user guide or instructions anywhere as to how to establish the PCIe connection in Linux after passive serial configuration Thanks Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration <p>It seems the FPGA isn't being detected as a PCIe device after passive serial configuration. To resolve this, ensure the correct PCIe drivers are installed and verify the device tree setup if using embedded Linux. You might also want to manually trigger a rescan and check the FPGA vendor's documentation for specific instructions on setting up PCIe detection. If the issue persists, contacting the FPGA manufacturer's support could provide further guidance.</p> Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Thank you corestar will try this out Regards Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Steve, Thanks for your confirmation, I will transition this support thread to community. I saw corestar have reply some workaround, perhaps you may refer to that and see if that able to help you or not. Any time in future, if you having any problem, you may file a new thread. Someone will be there to support you. If you need my support on PCIe IP, please mention my name "Wincent" so that this will route to my support Q. If your support experience falls below a 9 out of 10, I kindly request the opportunity to rectify it before concluding our interaction. If the issue cannot be resolved, please inform me via this forum page of the cause so that I can learn from it and strive to enhance the quality of future service experiences. Regards, Wincent_Altera p/s: If any answer from the community or Altera Support is helpful, please feel free to give the best answer or rate 9/10 survey. Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hello @Steve-Mowbray-ENL , I had the exact same problem. On a Raspberry Pi CM4, the solution outlined in the following link works by PeartreeStudios: PCIe Hot Swap The "rescan" idea does not work as you discovered. I have two scripts. bind_pci c ontains: echo "Bind PCIe Driver" sudo bash -c 'echo fd500000.pcie > /sys/bus/platform/drivers/brcm-pcie/bind' and unbind_pci contains: echo "Unbind PCIe Driver" sudo bash -c 'echo fd500000.pcie > /sys/bus/platform/drivers/brcm-pcie/unbind' These are clearly system dependent. But each time I update the FPGA firmware: Run unbind_pci if it was already bound Update the firmware via PS or JTAG Run bind_pci. Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Wincent Yes please close this ticket -- we are following up on Linux community since is specifically a Linux issue pertaining to FPGA... Regards Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Steve, I wish to follow up with you about this case. Do you have any further questions on this matter ? Else I would like to have your permission to close this forum ticket. Nevertheless, you can still response to the forum and I will be available to assist you. Regards, Wincent_Altera p/s: If any answer from the community or Altera Support is helpful, please feel free to give the best answer or rate 9/10 survey. Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Steve, without any manual FPGA programming steps does your system show the FPGA as a PCI device indicating that there is a working PCIe endpoint running on the FPGA after power on Linux boot? >> For step to programing the .sof file , you may refer to our official document >> https://www.intel.com/content/www/us/en/docs/programmable/784468/current/programming-hardware-sof-file.html >> if you need more clear picture about the how to check the PCIe link up status, programming the .sof , capturing .stp for the ltssm status, you may refer to >> https://community.intel.com/t5/FPGA-Intellectual-Property/How-to-run-PCIe-Gen-5-Design-Example-using-Altera-FPGA-device/m-p/1640988 >> Honestly I am authors of that document on link above, although is targeted Agilex 7 Gen 5 , but the fundamental and step would be 90 % above the same. You can list kernel boot options with the following: $cat /proc/cmdline >> Here you go, hope that able to help you If unconfigured can you see any trace of the FPGA PCIe endpoint using "lspci" command? >> First, I compile the design, then I program the .sof file. >> then perform sudo reboot. then lspci, detail step you may refer to the guide no2 mentioned above. >> If you found anything unclear , appreciate if you can provide your valuable feedback to me Regards, Wincent Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Wincent I think you misunderstood my power on Q1 and Q2 questions above -- without any manual FPGA programming steps does your system show the FPGA as a PCI device indicating that there is a working PCIe endpoint running on the FPGA after power on Linux boot? No working PCIe endpoint on the FPGA is the starting state for our system on power on since MSEL pins are set to passive serial. Thank you for confirming that $sudo reboot is used to initiate the PCI rescan -- we are hopeful we can use the same providing we identify the missing Linux kernel configurations You can list kernel boot options with the following: $cat /proc/cmdline Anything PCI would be of interest (eg perhaps pci=realloc) Regards Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Steve, When you power on your Linux system is the FPGA configured (by an active serial image) or completely unconfigured? >> I just use the default setting without any additional configuration. >> But in my cases it is "Passive Serial" >> If unconfigured can you see any trace of the FPGA PCIe endpoint using "lspci" command? >> Yes, the PCIe endpoint is able to enumerate after performing host reboot >> >> Signaltap shows link up in L0 stage as well >> What does "program the .sof via normal configuration" mean? Do you program the FPGA via JTAG perhaps? >> I use the default settings and JTAG cable , perhaps you can try USB blaster to program it. Is there any Linux scripts to run after configuration that are required to reset/rescan the FPGA PCIe endpoint? >> I dont use any linux scripts, I just perform reboot , using $ sudo reboot What Linux kernel boot options on your system? >> I am not sure this is helpful for you or not >> Is PCI hotplug enabled in your system BIOS? >> I believe Hotplug Flag is enabled by default in linux I share my success design .qar for you as a reference since that it is an "passive serial", perhaps it can give you some clue to resolve the problem that you facing now. The project is compile in Quartus v24.2. One things I realize that will causing your configuration fail would be, do you check your device "dip switch" https://cdrdv2-public.intel.com/670687/ug_a10-fpga-prod-devkit-683526-670687.pdf I am not sure what is your setting requirement, but you may try to set it as default settings if possible, I ever experience programming fail due to those switch toggling randomly in my board. Regards, Wincent Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Wincent Actually I have maybe a few questions about your previous response: >> thanks for confirming that, I have program the .sof via normal configuration in either CentOS 7 or Ubuntu 22.04 LTS >> I never seeing such scenario before When you power on your Linux system is the FPGA configured (by an active serial image) or completely unconfigured? If unconfigured can you see any trace of the FPGA PCIe endpoint using "lspci" command? What does "program the .sof via normal configuration" mean? Do you program the FPGA via JTAG perhaps? Is there any Linux scripts to run after configuration that are required to reset/rescan the FPGA PCIe endpoint? What Linux kernel boot options on your system? Is PCI hotplug enabled in your system BIOS? Regards Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Steve, Do you think you need any of my support at this moment ? If not , do I have your permission to close this thread ticket from my end ? Don't worry it just our internal procedure, this thread is still accessible by community support. If you discover a new finding related to FPGA , you may file a new thread in future. Our specialist will be there to support you. Regards, Wincent Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Wincent Thanks for your input thus far -- I will update this thread with the solution when we eventually discover one Regards Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Steve, I am 100% confident this is not FPGA side rather it is Linux side >> thanks for confirming that, I have program the .sof via normal configuration in either CentOS 7 or Ubuntu 22.04 LTS >> I never seeing such scenario before I am not sure if you can try to Perform BIOS update Update your Linux version into latest And see if you still monitor the same behavior or not. I trying to check any existing guide, but only option that I am able to find is https://www.intel.com/content/www/us/en/docs/programmable/683461/current/passive-serial-configuration.html But I think those are not so targeted to Linux specific use case. Perhaps those can provide you some hint to move forward. If the issue still not solved, and you confirm that the issue is from Linux sides, I suggest to try post the same question on Linux related forum. Please accept my apology for cannot help too much on this. Let me know if there is anything else on the PCIe IP that you feel I could provide my best assist. Regards, Wincent Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Wincent -- I am 100% confident this is not FPGA side rather it is Linux side -- we need guidance on the situation where FPGA is in unconfigured state Linux host boots up with the FPGA PCIe endpoint not present as to what the procedure in Linux is to rescan the PCIe bus to establish PCIe connectivity with the FPGA after Linux host completes passive serial configuration of the FPGA -- so far the possible avenues of investigation we have discovered are: Linux kernel boot options (pci=realloc etc) Linux native PCI hotplug Linux ACPI PCI hotplug Linux custom PCI register bashing (setpci etc and sysfs exposed functions...) This must be a pretty standard scenario when using any FPGA with Linux host that is based on the passive serial configuration scheme so I am hopeful there is some pre-existing guidance or experience with this but so far we have not been able to find anything detailed enough to replicate on our platform Regards Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Steve, I wondering , do you have any backup host to try this out ? I am trying to narrow this issue that is not specific host/CPU/BIOS related. Regards, Wincent Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Frank " Does the remove/configure/rescan sequence work for your with other configuration methods, e.g. JTAG or AS?" yes when we do an AS build and load the FPGA flash device with resulting JIC and also change the MSEL pins the FPGA PCIe device is enumerated and we can remove/rescan at will -- we can also change the MSEL pins while still running then do a passive serial write of our passive serial build RBF and remove/reset/rescan the PCIe devices into a working condition -- but what we want is power on in PS mode and do the FPGA configuration that way you mention the Terasic Arria TR10a-HL -- does that run with PC Linux and if so what is the sequence to rescan the PCI device when the FPGA is configured? Regards Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi, in my view it's unwanted to get the FPGA reconfigured unconditionally during PC reboot. This should only happen on power-on reset or explicite request. I can confirm that evaluation boards like Terasic Arria TR10a-HL are keeping FPGA configuration during PC reboot. This allows to check if the initial PCIe device enumeration works with your design. You previously mentioned you are not sure if NCONFIG is triggered in your hardware during reboot, but it should be easy to find out, e.g. by loading a test configuration over JTAG. Does the remove/configure/rescan sequence work for your with other configuarion methods, e.g. JTAG or AS? Regards Frank Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Wincent "Can you please check and ensure that those is been followed ?" Yes we have followed that document and seems to be working as evidenced by FPGA operating as expected after passive configuration eg LED's flashing and serial port connectivity etc except Linux doesn't enumerate the PCIe bus "Could you give some time more than 12 secs and see if the system is able to enumerate due to late configuration of FPGA via PS ?" I dont understand this request -- we can see FPGA in correct operation by observing LEDs and serial port connection but at this stage rescanning the PCIe bus doesnt work " Did you see the conf_done asserted high in the fail scenario ?" Yes conf_done is asserted high after passive serial configuration I am confident that the FPGA side of the configuration is working as expected -- the issue we have is getting the Linux PCI subsystem to connect to the FPGA after passive serial configuration -- if we do an active serial configuration FPGA build and change MSEL pins the FPGA gets PCIe enumerated correctly but we assume this is because the FPGA is ready in time for BIOS level enumeration. For passive serial configuration we are using the Linux shell commands: echo 1 >/sys/bus/pci/devices/0000:00:00.0/remove #do FPGA passive serial programming here... echo 1 > /sys/bus/pci/rescan Regards Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Steve, Since the Intel FPGA stores its configuration data in SRAM memory, and the PCIe HIP is part of the configuration that needs to happen, this gets configured each time the system boots up, or if a full reconfiguration happens. The Arria 10 handbook ( link ) briefly describes the configuration sequence and how to trigger a reconfiguration, in specific the configuration step is the one in which the data is written to the FPGA, however it does not explicitly mention what happen to the PCIe HIP block. The Configuring Altera FPGAs doc ( link ) describes in more detail the configuration and the reconfiguration process. Can you please check and ensure that those is been followed ? As far as I know, At the time the FPGA will be configured via PS, but that needs about 12 secs. To speed up, you may want to change the initialization to CvP. Still the minimal configuration to start the CvP will be done via PS and afterwards just to load the application via PCIe. Could you give some time more than 12 secs and see if the system is able to enumerate due to late configuration of FPGA via PS ? IF those still unable to solve your problem, I suspect to look it can be something is wrong with the connection on board or flash. Do you follow the pin connection guidelines? https://www.intel.com/content/www/us/en/docs/programmable/683814/current/intel-arria-10-gx-gt-and-sx-device-family.html . IF the design didn’t have the PCIe component that it is expected you are not seeing the FPGA as end point. Did you see the conf_done asserted high in the fail scenario ? Looking forward to hear back from you. Regards, Wincent Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Wincent -- reboot doesn't appear to force Linux to recognise the FPGA PCIe endpoint but there is the suspicion that the reboot causes the GPIO line tied to NCONFIG to go low... Regards Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Wincent -- passive serial programming seems to be working fine as indicated by other non PCIe behaviour is fully functional -- problem is specifically the FPGA endpoint is not recognised in the Linux system after programming and a PCIe rescan doesnt work Regards Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Thanks @FvM , for your help. Hi Steve, Do you try to reboot the system and see if the PCIe is able to enumerate ? Regards, Wincent Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi, not sure if the observed problem is failed configuration or only missing PCIe device enumeration by PC Bios. In the latter case, the problem may be simply too late configuration. Depending on Bios behaviour, rescan doesn't necessarily help, a reboot may be necessary. Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Steve, I am supporting PCIe IP area, based on your reply. I believe you try to convert the Programming Files RBF into passive serial configuration in order to run the design , am I get this right ? This is out of my support area actually, however I can try my best to assist In order to successful run the passive serial file please check below guide https://cdrdv2-public.intel.com/705024/ug-qpp-programmer-19-3-683039-705024.pdf Can you try to use JTAG mode and see if it can be configure successfully? If in JTAG mode it can be configure successfully, it can be something is wrong with the connection on board or flash. Do you follow the pin connection guidelines? https://www.intel.com/content/www/us/en/docs/programmable/683814/current/intel-arria-10-gx-gt-and-sx-device-family.html How is the MSEL setting? Hope my suggestion able to help you to move a step forward, Please accept my apology that this is out of my support area. IF your issue is still unable to solve, I suggest to file a new forum thread with title example "Arria 10: Passive Serial Configuration-programming failure" . So that our configuration team support specialist is able to look at this. Let me know if you have different thoughts. Regards, Wincent Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Wincent I dont see any Linux guidance related to the above link -- specifically it is the Linux steps required to get the PCIe endpoint correctly detected in the Linux PCIe master device after passive serial configuration that we need Regards Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Wincent Arria 10 (10AX066H4F34E3SG) and not using a design example but system is working fine in active serial mode but we are planning on using passive serial mode in product deployment Thanks Steve Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi Steve, For Passive serial configuration overall, you can refer to below link for the guide https://www.intel.com/content/www/us/en/support/programmable/support-resources/configuration/cfg-ps.html Hope that able to help you to move forward, let me know if there is any further clarification is needed. Regards, Wincent Replies: Re: FPGA passive serial configuration -- detect as PCIe device in Linux after configuration Hi, May I know which Device that you are using for this ? Also, are you using any PCIe design example ? Regards, Wincent