arrow_back

De-identifying DICOM Data with the Healthcare API

Join Sign in

De-identifying DICOM Data with the Healthcare API

1 hour 15 minutes 5 Credits

GSP626

Google Cloud selp-paced labs logo

Overview

In this lab you will discover and use the de-identification functionality of Cloud Healthcare API using Digital Imaging and Communications in Medicine (DICOM) data model.

In this lab, you will:

  • Gain a general understanding of Cloud Healthcare API and its role in managing healthcare data.

  • Learn how to create Cloud Healthcare API datasets and stores.

  • Import and Export DICOM data using the Cloud Healthcare API.

Setup and Requirements

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.

This hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.

To complete this lab, you need:

  • Access to a standard internet browser (Chrome browser recommended).
Note: Use an Incognito or private browser window to run this lab. This prevents any conflicts between your personal account and the Student account, which may cause extra charges incurred to your personal account.
  • Time to complete the lab---remember, once you start, you cannot pause a lab.
Note: If you already have your own personal Google Cloud account or project, do not use it for this lab to avoid extra charges to your account.

How to start your lab and sign in to the Google Cloud Console

  1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is the Lab Details panel with the following:

    • The Open Google Console button
    • Time remaining
    • The temporary credentials that you must use for this lab
    • Other information, if needed, to step through this lab
  2. Click Open Google Console. The lab spins up resources, and then opens another tab that shows the Sign in page.

    Tip: Arrange the tabs in separate windows, side-by-side.

    Note: If you see the Choose an account dialog, click Use Another Account.
  3. If necessary, copy the Username from the Lab Details panel and paste it into the Sign in dialog. Click Next.

  4. Copy the Password from the Lab Details panel and paste it into the Welcome dialog. Click Next.

    Important: You must use the credentials from the left panel. Do not use your Google Cloud Skills Boost credentials. Note: Using your own Google Cloud account for this lab may incur extra charges.
  5. Click through the subsequent pages:

    • Accept the terms and conditions.
    • Do not add recovery options or two-factor authentication (because this is a temporary account).
    • Do not sign up for free trials.

After a few moments, the Cloud Console opens in this tab.

Note: You can view the menu with a list of Google Cloud Products and Services by clicking the Navigation menu at the top-left. Navigation menu icon

Activate Cloud Shell

Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.

  1. In the Cloud Console, in the top right toolbar, click the Activate Cloud Shell button.

Cloud Shell icon

  1. Click Continue.

It takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. The output contains a line that declares the PROJECT_ID for this session:

Your Cloud Platform project in this session is set to YOUR_PROJECT_ID

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

  1. (Optional) You can list the active account name with this command:

gcloud auth list

(Output)

ACTIVE: * ACCOUNT: student-01-xxxxxxxxxxxx@qwiklabs.net To set the active account, run: $ gcloud config set account `ACCOUNT`
  1. (Optional) You can list the project ID with this command:

gcloud config list project

(Output)

[core] project = <project_ID>

(Example output)

[core] project = qwiklabs-gcp-44776a13dea667a6 For full documentation of gcloud, in Google Cloud, Cloud SDK documentation, see the gcloud command-line tool overview.

Create Healthcare Dataset

In this exercise you will use the UI to create a Cloud Healthcare API dataset

Under the Navigation Menu select Healthcare then Enable the API.

50723785440cb79.png

Once the API is enabled, in the Healthcare browser select Create Dataset.

Name the dataset dataset1 within region us-central1 and click Create.

5ed5bbccb4e7d6ef.png

Click Check my progress to verify the objective. Create Healthcare Dataset

Set up IAM Permissions

From the Navigation menu, go to IAM & admin > IAM.

cb4814c3a369910f.png

In the IAM page, select the Include Google-provided role grants box:

includeGoogleProvided.png

Edit the permissions for your Healthcare Service Agent by locating the service agent under the IAM list and selecting the pencil icon. The service account will have the Domain @gcp-sa-healthcare.iam.gserviceaccount.com.

dbeab1231f7f1a99.png

Click Add another role to add additional roles to the Healthcare Service Agent account. Click inside the Select a roll box and choose the following roles:

  • Cloud Storage > Storage Object Admin
  • Cloud Healthcare > Healthcare Dataset Administrator
  • Cloud Healthcare > Healthcare DICOM Editor

After all of the roles are added, select Save to commit your updates.

Enable data access logs on Cloud Healthcare

From the IAM & Admin menu, navigate to Audit Logs.

Scroll or use the filter box to locate Cloud Healthcare, then check the box next to it to select.

If the info panel isn't already open on the right side of the interface, click the Show Info Panel link.

show_info.png

Select Data Read and Data Write, then click Save

health_care_log_type.png

Click Check my progress to verify the objective. Set up IAM Permissions

Define variables needed

In Cloud Shell, export the variables needed for the lab:

export PROJECT_ID=`gcloud config get-value project` export REGION=us-central1 export DATASET_ID=dataset1 export DICOM_STORE_ID=dicomstore1

Create data stores

Data in Cloud Healthcare API datasets and stores can be accessed and managed using a REST API that identifies each store using its project, location, dataset, store type and store name. This API implements modality-specific standards for access that are consistent with industry standards for that modality. For example, the Cloud Healthcare DICOM API natively provides operations for reading DICOM studies and series that are consistent with the DICOMweb standard, and supports the DICOM DIMSE C-STORE protocol via an open-source adapter.

Call the API to create a DICOM store:

gcloud beta healthcare dicom-stores create $DICOM_STORE_ID --dataset=$DATASET_ID --location=$REGION

The server returns a path to the newly created store.

Users can also use the curl utility to issue Cloud Healthcare API calls. curl is pre-installed in your Cloud Shell machine. By default, curl does not show HTTP status codes or session-related information; if you would like to see this information please add the -v option to all commands in this tutorial.

Try creating a secondary FHIR store by using the below command:

curl -X POST \ -H "Authorization: Bearer "$(sudo gcloud auth print-access-token) \ -H "Content-Type: application/json; charset=utf-8" \ "https://healthcare.googleapis.com/v1beta1/projects/$PROJECT_ID/locations/$REGION/datasets/$DATASET_ID/dicomStores?dicomStoreId=dicomstore2"

Operations that access a modality-specific store use a request path that is comprised of two pieces: a base path, and a modality-specific request path. Administrative operations—which generally operate only on locations, datasets and stores—may only use the base path. Data modality-specific retrieval operations use both the base path (for identifying the store to be accessed) and request path (for identifying the actual data to be retrieved).

Click Check my progress to verify the objective. Create data stores

If this check fails, wait a minute and try again. It often takes a minute or two for the import operation to be logged.

Import to DICOM Datasets

In this section you will be importing data from the NIH Chest x-ray data set to a DICOM store. For more information on the public dataset, visit the documentation: https://cloud.google.com/healthcare/docs/resources/nih-chest

Call the API to use the import functionality:

gcloud beta healthcare dicom-stores import gcs $DICOM_STORE_ID --dataset=$DATASET_ID --location=$REGION --gcs-uri=gs://spls/gsp626/LungCT-Diagnosis/R_004/*

Click Check my progress to verify the objective. Import to DICOM Datasets

Configure OHIF Viewer

The Open Health Imaging Foundation (OHIF) Viewer is an open source, web-based, medical imaging viewer. You will use OHIF Viewer in this lab to view your DICOM dataset.

The following steps will walk through setting up OHIF Viewer to view your dataset:

  1. First, select APIs & Services > OAuth Consent Screen from the Navigation Window to create an OAuth Consent screen:

oauthConsent.png

At the OAuth Consent Screen, select Internal and click Create:

consentScreen.png

Fill out the following on the Edit app registration window:

  • App name: QL-de-identify
  • User support email: YOUR STUDENT EMAIL (this is provided by the lab)
  • Developer contact information: YOUR STUDENT EMAIL (same value as user support email)

editRegister.png

Click Save and Continue.

At the Scopes tab, click the Add or Remove Scopes button.

Scroll to the bottom of the pop-up window to the Manually add scopes section.

Add the following scopes:

https://www.googleapis.com/auth/cloudplatformprojects.readonly https://www.googleapis.com/auth/cloud-healthcare

scopes.png

Click Add to table and then click Update.

Scroll to the bottom of the Scopes tab and click Save and Continue.

  1. Next, you'll need an OAuth Client ID to connect OHIF Viewer to your Cloud Healthcare resources.

Select Credentials from the APIS & Services menu:

credentials.png

In the Credentials page, click + Create Credentials > OAuth Client ID:

oauthClientID.png

For your Application Type, choose Web application.

You will need to return to your client ID and fill out the domains once your OHIF Viewer application has been launched.

So, for now, leave everything as default and click Create.

clientIDInfo.png

You'll now the see your Client ID and Client Secret in the following window.

Click OK to close the window.

  1. Now, deploy the OHIF Viewer container to Cloud Run and connect it with your OAuth Client ID.

To simplify the setup, the OHIF Viewer docker image already exists in container registry in a project you have access to, so you can directly deploy the container to Cloud Run.

In Cloud Shell, deploy the OHIF Viewer container to Cloud Run with this command substituting PASTE-CLIENT-ID-HERE with the Client ID of the OAuth Client you just created:

gcloud run deploy ohif-viewer --image=gcr.io/qwiklabs-resources/ohif-viewer:latest --platform=managed --region=us-central1 --allow-unauthenticated --set-env-vars=CLIENT_ID=[PASTE-CLIENT-ID-HERE] --max-instances=3 You can view and copy your Client ID in the Credentials tab: copyClientID.png

If asked to enable the Cloud Run API, enter y and continue.

Once your Cloud Run deployment completes, you will be given a unique service URL that looks similar to this:

Service URL: https://ohif-viewer-ratpkirjdq-uc.a.run.app
  1. You can now return to your OAuth Client ID and update the domains with this Service URL.

If you're not still on the Credentials page, select APIs & Services > Credentials from the Navigation Menu in your Cloud Console.

Edit your Client ID by clicking the pencil icon.

editClient.png

Add your unique service URL to Authorized Javascript Origins.

Add your unique service URL + /callback to Authorized Redirect URIs.

addDomains.png

Click Save.

Using De-identification

De-identification (redacting or transformation) of sensitive data elements is often an important step in pre-processing healthcare data so that it can be made available for analysis, machine learning models, and other use cases. Cloud Healthcare API has the capability to de-identify data stored in the service, facilitating analysis by researchers or machine learning analysis for advanced anomaly scans.

Step 1

First, navigate to the service URL of your ohif-viewer Cloud Run app and sign in using your lab credentials. If you've lost track of your service URL, you can find it again with this command:

gcloud run services list --platform managed

Once on the OHIF-Viewer page, select your Project ID for the Project:

ohifFirstPage.png

Select us-central1 for the location.

Select dataset1 for your dataset.

Select dicomstore1 in the DICOM Store window.

You'll see one entry, R_004 with info for its ID number, Study Date, and Description:

ohifSample.png

Click on the entry to inspect it further and view the associated images.

This dataset contains pre-surgery images of a chest. You can scroll through them to view them all:

ohifScroll.gif

When you're done looking at it, press the Back button on your browser to return to the OHIF-Viewer main menu.

Next, you will de-identify this dataset.

Step 2

Navigate back to Cloud Shell and issue the following request to de-identify the dataset:

curl -X POST \ -H "Authorization: Bearer "$(gcloud auth print-access-token) \ -H "Content-Type: application/json; charset=utf-8" \ --data "{ 'destinationDataset': 'projects/$PROJECT_ID/locations/$REGION/datasets/de-id', 'config': { 'dicom': { 'filterProfile': 'ATTRIBUTE_CONFIDENTIALITY_BASIC_PROFILE' }, 'image': { 'textRedactionMode': 'REDACT_NO_TEXT' } } }" "https://healthcare.googleapis.com/v1beta1/projects/$PROJECT_ID/locations/$REGION/datasets/$DATASET_ID:deidentify"

With our small dataset, this operation will be done quickly, but on a larger dataset this operation can take a few minutes. You can issue a rest request to check the status of a long running operation, replacing with the operations ID issued in the previous output.

curl -X GET \ "https://healthcare.googleapis.com/v1beta1/projects/$PROJECT_ID/locations/$REGION/datasets/$DATASET_ID/operations/<operation-id>" \ -H "Authorization: Bearer "$(sudo gcloud auth print-access-token) \ -H 'Content-Type: application/json; charset=utf-8'

If you see "done": true in the output of the previous command, you can be sure that your operation is complete.

Step 3

Once the operation is complete a new de-id dataset will appear on the Healthcare UI page in the Console.

Confirm the identifiable information has been redacted by returning to your OHIF-Viewer browser tab and selecting the Change DICOM Store button:

changeStore.png

In the window that pops up, select your Qwiklabs Project ID as the Project.

Select us-central1 for the location.

Select de-id as the dataset.

Select dicomstore1 for the DICOM Store.

You'll now see one entry in the DICOM Store, but the outward facing information/tags have been removed:

redacted.png

Select the entry to confirm it's the same images copied from the previous dataset but with most of its information removed.

Click Check my progress to verify the objective. Using De-identification

Converting DICOM Images

From the Navigation menu, navigate to Storage.

Click Create bucket.

Fill out the first box with a unique name and click Create.

Using Cloud Shell export the variable for your newly created bucket, replacing with you bucket's name:

export BUCKET_ID=<name of bucket>

Now you can export the DICOM images into JPEG or PNG using a gcloud command.

Export the DICOM images into JPEG:

gcloud beta healthcare dicom-stores export gcs $DICOM_STORE_ID --dataset=$DATASET_ID --gcs-uri-prefix=gs://$BUCKET_ID/ --mime-type="image/jpeg; transfer-syntax=1.2.840.10008.1.2.4.50"

OR

Export the DICOM images into PNG:

gcloud beta healthcare dicom-stores export gcs $DICOM_STORE_ID --dataset=$DATASET_ID --gcs-uri-prefix=gs://$BUCKET_ID/ --mime-type="image/png"

In the Console, from the Navigation menu navigate to Storage and click on your bucket.

Select a folder, click on an image, then click on the Link URL. This will download the image.

You can check the file extension to verify your file is correct or click the image to view.

Click Check my progress to verify the objective. Converting DICOM Images

Lab review

Cloud Healthcare API provides a comprehensive facility for ingesting, storing, managing, and securely exposing healthcare data in FHIR, DICOM, and HL7 v2 formats. Using Cloud Healthcare API, you can ingest and store data from electronic health records systems (EHRs), radiological information systems (RISs), and custom healthcare applications. You can then immediately make that data available to applications for analysis, machine learning prediction and inference, and consumer access.

Cloud Healthcare API enables application access to healthcare data via widely-accepted, standards-based interfaces such as FHIR STU3 and DICOMweb. These APIs allow data ingestion into modality-specific data stores, which support data retrieval, update, search and other functions using familiar standards-based interfaces.

Further, the API integrates with other capabilities in Google Cloud through two primary mechanisms:

  • Cloud Pub/Sub, which provides near-real-time updates when data is ingested into a Cloud Healthcare API data store, and
  • Import/export APIs, which allow you to integrate Cloud Healthcare API into both Google Cloud Storage and Google BigQuery.

Using Cloud Pub/Sub with Google Cloud Functions enables you to invoke machine learning models on healthcare data, storing the resulting predictions back in Cloud Healthcare API data store. A similar integration with Cloud Dataflow supports transformation and cleansing of healthcare data prior to use by applications.

To support healthcare research, Cloud Healthcare API offers de-identification capabilities for FHIR and DICOM. This feature allows customers to share data with researchers working on new cutting-edge diagnostics and medicines.

Congratulations

In this lab you:

  • Gained a general understanding of Cloud Healthcare API and its role in managing healthcare data.
  • Learned how to create datasets and stores for FHIR and DICOM data.
  • Imported FHIR and DICOM data.

Healthcare_125.png

Finish Your Quest

This self-paced lab is part of the Qwiklabs [Cloud Healthcare API] Quest. A Quest is a series of related labs that form a learning path. Completing a Quest earns you a badge to recognize your achievement. You can make your badge (or badges) public and link to them in your online resume or social media account. Enroll in either Quest and get immediate completion credit if you've taken this lab. See other available Qwiklabs Quests.

Take your next lab

Continue your quest with Ingesting FHIR Data with the Healthcare API or try one of these suggestions:

End your lab

When you have completed your lab, click End Lab. Qwiklabs removes the resources you’ve used and cleans the account for you.

You will be given an opportunity to rate the lab experience. Select the applicable number of stars, type a comment, and then click Submit.

The number of stars indicates the following:

  • 1 star = Very dissatisfied
  • 2 stars = Dissatisfied
  • 3 stars = Neutral
  • 4 stars = Satisfied
  • 5 stars = Very satisfied

You can close the dialog box if you don't want to provide feedback.

For feedback, suggestions, or corrections, please use the Support tab.

Manual Last Updated: January 23, 2021
Lab Last Tested: January 23, 2021

Copyright 2022 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.