"【Evaluation】Differential Privacy and Federated Learning in Medical Data"

2024-04-30

More Protection Needed for Sensitive Data

In the era of large-scale language models trained from the public internet, the demand for data privacy seems to have generally relaxed, regardless of the actual intellectual property acknowledged by their respective company leaders.

However, when it comes to patient data, which is our health records, there is a more sensitive parallel world that undoubtedly requires much more protection.

In addition, regulations around the world are constantly strengthening, including data protection regulations, including artificial intelligence, which are also becoming more stringent.

There are obvious ethical reasons for this, which we don't need to explain, but from a regulatory and legal perspective at the enterprise level, pharmaceutical companies, laboratories, and hospitals must use state-of-the-art technology to protect patient data privacy.

Joint Examples Can Provide Assistance

Joint analysis and learning are the best choices for analyzing and training patient data without accessing any raw data.

For example, in terms of joint analysis, this means that we can obtain the correlation between blood sugar and patient body mass index without accessing any raw data that may lead to patient re-identification.

As for machine learning, taking diagnostic learning as an example, by training models on patient images, malignant changes in patient tissues can be detected, such as detecting early stages of cancer. This is a life-saving application of machine learning. At the hospital level, local images and labels specified by professional radiologists are used for local model training, and then aggregated to merge all these local models into a more general model. This process is repeated tens or hundreds of rounds to improve the performance of the model.

For each hospital, a well-trained model will greatly benefit them, enabling them to detect diseases in future patients with a higher probability. This is a win-win situation for everyone, especially patients.

Of course, there are various alliance network topologies and model aggregation strategies, but in this article, we try to focus on typical examples.

Building Trust with Technology

It is believed that a large amount of clinical data is not utilized because data owners are unwilling to share data with partners (which is reasonable).

Joint learning is a key strategy for building trust with technology, rather than relying solely on contracts, ethical beliefs of specific employees, and cooperation of organizations forming the consortium.

Firstly, the data remains at the source and has never left the hospital, nor is it centralized in a single potentially vulnerable location. The federated approach means that there are no external data copies, which may be difficult to remove after the research is completed.

By adopting various technologies that follow the principles of defense in depth, this technology can prevent access to raw data. Each technology can reduce the risk of data exposure and patient re-identification by tens or thousands of times. All of this makes it economically infeasible to discover or reconstruct the original data.

We first minimize the data by exposing only the necessary attributes to the locally running machine learning agent, stripping off PII data, and using anonymization techniques.

Then, the local nodes protect their data from so-called "overly curious data scientists" by running code and operations on their data that are only accepted by local data owners. For example, the model training code deployed locally in the hospital is packaged as a software package that is allowed or disallowed by local data owners. Remote data scientists cannot send any code to remote nodes, as this would allow them to return the original data.

Is the Model Private Enough?

Assuming all these protection layers are in place, the security of the model weights themselves is still a concern.

The AI community's attention to machine learning models is growing, considering them as super compressions of data, not as black box operations as previously thought, and revealing more underlying data information than previously thought.

This means that as long as there is enough skill, time, energy, and powerful hardware, a motivated adversary can attempt to reconstruct the original data or at least prove with a high probability that a patient belongs to the population used to train the model (membership inference attack (MIA)). There are also other types of attacks, such as extraction, reconstruction, and evasion.

What's worse, the advances in generative AI that we all admire and benefit from provide new and more effective techniques for image reconstruction (e.g., lung scans of patients). The idea we all use to generate images on demand can also be used by adversaries to reconstruct the original images from MRI/CT scanners. Other data patterns, such as tabular data, text, sound, and video, can now also be reconstructed using generative AI.

Differential Privacy to the Rescue

Differential privacy (DP) algorithms promise that we can trade off some accuracy of the model for stronger resistance to inference attacks. This is another privacy-utility trade-off worth considering.

Differential privacy means that in practice, we add a very specific type of noise and clipping that results in a good ratio of privacy gain to accuracy loss.

It can be as simple as minimal effective Gaussian noise, but today we have seen the development of more complex algorithms such as sparse vector technique (SVT), Opacus library implemented as differentially private stochastic gradient descent (DP-SGD), and ancient libraries based on Laplace noise (i.e., PyDP).

By the way, we all benefit from this technology without even realizing its existence, and it is happening now. Telemetry data we obtain from mobile devices (Apple iOS, Google Android) and desktop operating systems (Microsoft Windows) are being used to train models using differential privacy and federated learning algorithms without sending raw data from our devices. This technology has been around for many years.

Now, the adoption rate for other use cases is also increasing, including our favorite island-style federated learning use cases, where participants from different organizations and companies have relatively less involvement but possess a large amount of data.

Differential privacy is not unique to federated learning. However, different strategies are applied to data privacy in federated learning scenarios, and the chosen algorithms are also different. Different algorithms are more effective for federated learning settings, local differential privacy (LDP), and centralized data processing.

In the context of federated learning, we expect that after applying differential privacy, the accuracy of the model will decrease, but we still (to some extent) expect the performance of the model to be better than local models without federated aggregation. Therefore, despite the addition of noise and clipping (DP), the federated model should still maintain its advantage.

Differential privacy was originally applicable to source data (local differential privacy (LDP)).

There are also cases of federated learning within partner networks, where partners have full data access rights and less attention to the level of data protection, so DP may not be needed at all.

On the other hand, when the model is to be shared with or commercially sold to the outside world, applying data protection to the global model is also a good idea.

Actual Experimental Results

In Roche's Federated Open Science team, NVIDIA Flare is our preferred federated learning tool because it is the most mature open-source federated framework on the market. We have also collaborated with the NVIDIA team on the future development of NVIDIA Flare and are pleased to help improve this already excellent federated learning solution.

We tested three different DP algorithms: