As technology continues to advance, so do the capabilities of artificial intelligence and machine learning (AI/ML) software. These advancements are making a significant impact on the healthcare industry, from improving patient outcomes to enhancing the overall quality of care. However, as with any new technology, it's important to ensure that it's used safely and effectively. That's where the new draft guidance for Marketing Submission Recommendations for a Predetermined Change Control Plan for AI/ML-Enabled Software from the FDA comes in.
The new draft guidance provides recommendations for AI/ML-enabled medical devices that are intended to be modified by its manufacturer over time. It's based on a number of public meetings and workshops on artificial intelligence and machine learning, and it’s consistent with the White House's new Blueprint for an AI Bill of Rights. The guidance describes what type of additional content should be included in premarket authorization applications for AI/ML-enabled medical devices, including describing the contents that FDA expects in a Predetermined Change Control Plan (PCCP).
The PCCP is a plan that manufacturers should include as part of their premarket authorization application, whether that is the 510(k) pathway, De Novo, or PMA. It includes a Description of Modifications, a Modification Protocol, and an Impact Assessment. The Description of Modifications is an essential part of the PCCP, as it draws a range of authorized specifications around the initial characteristics and performance of the device that can be implemented without a new marketing submission. This helps to ensure that any modifications made to the device are within the scope of the original authorization and are consistent with the device's intended use.
The description of the modifications that you provide should be specific and must be able to be verified and validated. Each modification should be linked to a specific performance evaluation activity, which will be described in the Modification Protocol section of the PCCP - more about that later. The Description of Modifications is also where you describe how modifications will be introduced to devices that are currently released - whether updates will be automatic or manually performed, or whether all devices will be updated simultaneously or if only a subset of devices will be updated.
As a rule, modifications described in the PCCP are intended to improve the safety or effectiveness of the AI/ML-enabled device. This includes a few different types of modifications, like those to improve model performance, changes to inputs, or limited modifications to a device’s use (such as a use within a specific subpopulation). However, any modifications must still maintain the intended use and indications for use for your device. Examples of the types of modifications that FDA expects include modifying a skin lesion identifying software that was validated on a specific camera model to include use with other camera hardware that meets minimum specifications.
The PCCP also should include a Modification Protocol section, which notes how the modifications will be implemented. The FDA expects that the Modification Protocol section of your PCCP contains several components, including data management practices, re-training practices, performance evaluations, and update procedures.
The data management practices section includes describing how you obtain your training and testing data, how that data is used, whether or not that data adequately represents identifiable subpopulations in your intended use population, how you ensure you’re not overfitting, and other data management practices. The re-training practices section allows FDA to determine if your changes are implemented following appropriate, well-defined practices. The performance evaluations included in the PCCP should include a description of when performance evaluations are triggered, a description of the statistical analysis to be performed, and acceptance criteria. The update procedures section should include information about any verification or validation testing that will be performed, when and how updates will be implemented, how these updates are communicated to users, and how devices are monitored in the field.
It’s important to maintain traceability between the descriptions of modifications to your device and the Modification Protocol sections described above. This can be accomplished using a traceability matrix, with each row representing a described modification. Each column represents a section from the Modification Protocols - data management practices, re-training practices, performance evaluation, and update procedures.
In the PCCP, the Impact Assessment is the assessment of the benefits and risks (and their associated mitigations) introduced by the modifications described in the PCCP. The Impact Assessment should compare the version of the device with each modification implemented to the device without any modifications implemented. The comparison should include an assessment of the benefits and risks of each modification, including risks of social harm, and discuss how the activities described in the Modification Protocol ensure the ongoing safety and effectiveness of the device. It’s also important to consider how the implementation of one modification impacts other modifications, as well as the collective impact of all modifications.
By following this new guidance including a PCCP as part of your premarket authorization application, manufacturers can ensure that their medical device software continues to function safely and effectively and ensure they’re following FDA regulations. Contact us if you’re ready to learn more about FDA regulations about artificial intelligence and machine learning.
