Post Market Monitoring System (PMS)
Last updated
Last updated
This compliance category contains details about the Post Market Monitoring System required to be put in place for the FDA AI/ML based SaMD.
This compliance category support the secion 7.5 fo the IMDRF/SaMD N23, which states:
Measurement of quality characteristics of software products and processes is used to manage and improve product realization and use. An effective measurement of key factors, often associated with issues related to risk, can help identify the capabilities needed to deliver safe and effective SaMD. Opportunities to monitor, measure, and analyze for improvement exist before, during, and after SaMD lifecycle processes, activities and tasks, and are completed with the intent to objectively demonstrate the quality of the SaMD. Post market surveillance including monitoring, measurement and analysis of quality data can include logging and tracking of complaints, clearing technical issues, determining problem causes and actions to address, identify, collect, analyze, and report on critical quality characteristics of products developed. For SaMD, monitoring to demonstrate through objective measurement that processes are being followed does not itself guarantee good software, just as monitoring software quality alone does not guarantee that the objectives for a process are being achieved.
Point four in the FDA AI/ML discussion paper revolved around the transparency and real-world performance monitoring of AI/ML-based SaMD. It states:
To fully adopt a TPLC (Total Product Lifecycle) approach in the regulation of AI/ML-based SaMD, manufacturers can work to assure the safety and effectiveness of their software products by implementing appropriate mechanisms that support transparency and real-world performance monitoring. Transparency about the function and modifications of medical devices is a key aspect of their safety. This is especially important for devices, like SaMD that incorporate AI/ML, which change over time. Further, many of the modifications to AI/ML-based SaMD may be supported by collection and monitoring of real-world data. Gathering performance data on the real-world use of the SaMD may allow manufacturers to understand how their products are being used, identify opportunities for improvements, and respond proactively to safety or usability concerns. Real-world data collection and monitoring is an important mechanism that manufacturers can leverage to mitigate the risk involved with AI/ML-based SaMD modifications, in support of the benefit-risk profile in the assessment of a particular AI/ML-based SaMD.
Through this framework, manufacturers would be expected to commit to the principles of transparency and real-world performance monitoring for AI/ML-based SaMD. FDA would also expect the manufacturer to provide periodic reporting to FDA on updates that were implemented as part of the approved SPS (SaMD Pre-Specification) and ACP (Algorithm Change Protocol), as well as performance metrics for those SaMD. This commitment could be achieved through a variety of mechanisms.
From the guiding principles published by the FDA as GMLP, this category support the principle:
Principle 10. Deployed Models Are Monitored for Performance and Re-training Risks Are Managed: Deployed models have the capability to be monitored in “real world” use with a focus on maintained or improved safety and performance. Additionally, when models are periodically or continually trained after deployment, there are appropriate controls in place to manage risks of overfitting, unintended bias, or degradation of the model (for example, dataset drift) that may impact the safety and performance of the model as it is used by the Human-AI team.
This compliance category supports the following controls/checks: