arrow_back

Ingesting DICOM Data with the Healthcare API

Join Sign in

Ingesting DICOM Data with the Healthcare API

1 hour 15 minutes 1 Credit

GSP615

Google Cloud selp-paced labs logo

Overview

In this lab you will discover and use the basic functionality of Cloud Healthcare API using Digital Imaging and Communications in Medicine (DICOM) data model.

Healthcare API introduction

Cloud Healthcare API provides a managed solution for storing and accessing healthcare data in Google Cloud Platform (GCP), providing a critical bridge between existing care systems and applications hosted on Google Cloud. Using the API, you can unlock significant new capabilities for data analysis, machine learning and application development, and use these capabilities to build the next generation of healthcare solutions.

The API is comprised of three modality-specific interfaces that implement key industry-wide standards for healthcare data:

  • FHIR, an emerging standard for health data interchange
  • HL7v2, the most widely adopted method for health systems integration
  • DICOM, the dominant standard for radiology and imaging-related disciplines

Each interface is backed by a standards-compliant data store that provides read, write, search, and other operations on the data.

The Cloud Healthcare API provides a number of key features that are critical to bridging current technologies to the next generation of healthcare systems and applications:

  • Standards conformance - Google supports the use of standards-based interoperability through its participation in a number of healthcare standards bodies. In the Cloud Healthcare API each modality-specific data store and its associated API is substantially conformant with its relevant standard. For example, FHIR stores implement STU3, the current version of the FHIR specification, and DICOM stores implement DICOMweb, a web-based standard for exchanging medical images. In future updates, we expect to support additional versions of these specifications as well as the ability to request a resource in a different version than its canonical representation.
  • Compliance with privacy regulations - GCP provides detailed guidance regarding how it supports compliance with HIPAA in the US, the PIPEDA in Canada, and other global privacy standards at cloud.google.com/security/compliance.
  • Data location control - The Cloud Healthcare API treats data location as a core component of the API. You have the option to select the storage location for each dataset from a list of currently available locations which correspond to distinct geographic areas aligned with GCP's regional structure. Future GCP regions will allow for the distribution of storage across wider geographic areas.
  • Security - The Cloud Healthcare API security model is based on Google's proven Identity and Access Management (IAM) system. IAM's fine-grained permissions give you complete control over access to your healthcare data. In addition, we've created open-source proxies for our powerful Apigee API Management system, which provides comprehensive threat detection and traffic management capabilities that allow you to securely expose sensitive ePHI with patient and provider applications.
  • Bulk import and export - The Cloud Healthcare API's DICOM and FHIR modalities support bulk import and export of data, making it easier to transfer data via the Cloud Storage system.
  • De-identification - De-identification support for DICOM is available, making it much easier to redact patient information from studies for research and other purposes. The de-identification process operates on a data store basis.
  • Auditability - Both administrative and data access requests to the Cloud Healthcare API can be audited. Logs are available through Google Cloud's Stackdriver hybrid monitoring system.
  • High availability - Availability for mission-critical scenarios is made possible through Google Cloud's robust and highly redundant infrastructure.

For many applications, the Cloud Healthcare API can provide a modern alternative to legacy stacks implementing DICOM, HL7v2 or FHIR STU3 standards, simplifying data integration with existing systems and enabling the application developers to focus on their differentiating features such as UX and intelligence.

In this lab, you will:

  • Gain a general understanding of the Cloud Healthcare API and its role in managing healthcare data.
  • Learn how to create Cloud Healthcare API datasets and stores..
  • Import and Export DICOM data using the Cloud Healthcare API.

Healthcare API concepts

To get the most out of the Cloud Healthcare API, there are a few key concepts you'll want to understand. The information below should give you a good sense of Cloud Healthcare API capabilities, but you can find more details in the documentation.

General structure of the Cloud Healthcare API

The Cloud Healthcare API exposes interfaces that enable you to perform different types of functions:

  • Administrative functions, such as creating or listing datasets and stores that will contain your data;
  • Data access functions that allow you to create, update, delete and search the data stored in Cloud Healthcare API, or to perform bulk import and export operations;
  • Security functions that allow you to impose access controls on data stored in Cloud Healthcare API;
  • De-identification functions that allow you to replace ePHI with anonymized data, or to obfuscate ePHI so that it cannot be used;
  • Metadata functions, such as retrieval of a FHIR capabilities statement for the FHIR API.

These functions may vary slightly depending on the modality of data (FHIR, HL7 v2 or DICOM) being operated on. For example, data retrieval operations against an FHIR data store use an API that conforms to the FHIR standard, but data retrieval operations against an HL7 v2 store use operations better suited to operating on HL7v2-structured data.

Datasets and stores

All Cloud Healthcare API usage occurs within the context of a Google Cloud Platform project. Projects form the basis for creating, enabling, and using all GCP services including managing APIs, enabling billing, adding and removing collaborators, and managing permissions for GCP resources. Cloud Healthcare API can be used in one or many GCP projects, as appropriate; this flexibility allows you to separate production from non-production usage, for example, or to segregate applications and resources in order to better manage access or accommodate different development lifecycles.

Within a project, data ingested through Cloud Healthcare API is stored in a dataset, which resides in a geographic location corresponding to a specific GCP region. You use the Cloud Healthcare API's administrative functions to create a dataset in a particular location; doing so facilitates implementation of data location requirements for the countries in which your applications provide services. For example, you can choose to create a dataset in GCP's "us-central1" region for US-based applications, or in an EU or UK region for applications serving those customers. This level of location control is also available in other GCP products, which can be combined with Cloud Healthcare API to create a complete application architecture. A list of generally available GCP products and the regions in which they are implemented can be found here.

Because each healthcare data modality has different structural and processing characteristics, datasets are split into modality-specific stores. A single dataset can contain one or many stores, and those stores can all service the same modality or different modalities as application needs dictate. Using multiple stores in the same dataset might be appropriate if a given application processes different types of data, for example, or if you'd like to be able to separate data according to its source hospital, clinic, department, etc. An application can access as many datasets or stores as its requirements dictate with no performance penalty, so it's important to design your overall dataset and store architecture to meet the organization's broad goals for locality, partitioning, access control, and so on.

The diagram below illustrates two datasets in a GCP project, each of which contains multiple stores.

healthcareAPI_data_model.png

There are many ways to structure datasets and stores. As you design systems that use the Cloud Healthcare API, you may want to take the following into consideration:

  • Security and access control: Rules can be defined at both a dataset and store level, but you may choose to group all data for a particular application into the same dataset, and set access control rules such that only that application can access the dataset.

  • Application requirements: An application processing different types of data may have all of its data for all modalities in a single dataset.

  • Source systems: Often, the structure of healthcare data can vary according to the source system and modality. Separating data for different source systems into their own datasets may facilitate processing.

  • Intended use: Data from different systems can have different intended uses, such as research, analytics or machine learning predictions. Grouping data by intended use may facilitate ingestion into the target system.

  • Separating ePHI from de-identified data: Cloud Healthcare API data de-identification functions read from a source dataset and write the output into a new dataset that you specify. If you are preparing data to be used by researchers, for example, this approach to de-identifying data may be a consideration in how you use datasets to segregate data.

Setup and Requirements

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.

This hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.

To complete this lab, you need:

  • Access to a standard internet browser (Chrome browser recommended).
Note: Use an Incognito or private browser window to run this lab. This prevents any conflicts between your personal account and the Student account, which may cause extra charges incurred to your personal account.
  • Time to complete the lab---remember, once you start, you cannot pause a lab.
Note: If you already have your own personal Google Cloud account or project, do not use it for this lab to avoid extra charges to your account.

How to start your lab and sign in to the Google Cloud Console

  1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is the Lab Details panel with the following:

    • The Open Google Console button
    • Time remaining
    • The temporary credentials that you must use for this lab
    • Other information, if needed, to step through this lab
  2. Click Open Google Console. The lab spins up resources, and then opens another tab that shows the Sign in page.

    Tip: Arrange the tabs in separate windows, side-by-side.

    Note: If you see the Choose an account dialog, click Use Another Account.
  3. If necessary, copy the Username from the Lab Details panel and paste it into the Sign in dialog. Click Next.

  4. Copy the Password from the Lab Details panel and paste it into the Welcome dialog. Click Next.

    Important: You must use the credentials from the left panel. Do not use your Google Cloud Skills Boost credentials. Note: Using your own Google Cloud account for this lab may incur extra charges.
  5. Click through the subsequent pages:

    • Accept the terms and conditions.
    • Do not add recovery options or two-factor authentication (because this is a temporary account).
    • Do not sign up for free trials.

After a few moments, the Cloud Console opens in this tab.

Note: You can view the menu with a list of Google Cloud Products and Services by clicking the Navigation menu at the top-left. Navigation menu icon

Activate Cloud Shell

Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.

  1. In the Cloud Console, in the top right toolbar, click the Activate Cloud Shell button.

Cloud Shell icon

  1. Click Continue.

It takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. The output contains a line that declares the PROJECT_ID for this session:

Your Cloud Platform project in this session is set to YOUR_PROJECT_ID

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

  1. (Optional) You can list the active account name with this command:

gcloud auth list

(Output)

ACTIVE: * ACCOUNT: student-01-xxxxxxxxxxxx@qwiklabs.net To set the active account, run: $ gcloud config set account `ACCOUNT`
  1. (Optional) You can list the project ID with this command:

gcloud config list project

(Output)

[core] project = <project_ID>

(Example output)

[core] project = qwiklabs-gcp-44776a13dea667a6 For full documentation of gcloud, in Google Cloud, Cloud SDK documentation, see the gcloud command-line tool overview.

Create Healthcare dataset

From the Console, go to Navigation menu > Healthcare.

Healthcare_menu.png

Enable the API.

Now click Create Dataset.

Name the dataset "dataset1" and select the us-central1 region.

healthcare_name_dataset.png

Then click Create.

Click Check my progress to verify the objective. Create Healthcare Dataset

Set up IAM Permissions

From the Navigation menu select IAM & admin > IAM.

IAM-ADMIN_iam.png

At the top of the Permissions table, look for the Include Google-provided role grants box and click it:

google-provided-role-grants.png

Edit the permissions for your Cloud Healthcare Service Agent by locating the service agent under the IAM list and selecting the pencil icon. The service account will have the Domain @gcp-sa-healthcare.iam.gserviceaccount.com.

dbeab1231f7f1a99.png

Click add another role in the dialog and then selecting the appropriate role.

  • BigQuery >BigQuery Admin
  • Cloud Storage >Storage Object Admin
  • Cloud Healthcare >Healthcare Dataset Administrator

When all of the roles have been added, select Save to commit your updates.

Click Check my progress to verify the objective. Set up IAM Permissions

Define Variables Needed

In Cloud Shell run the following to set variables needed for this lab:

export PROJECT_ID=`gcloud config get-value project` export REGION=us-central1 export DATASET_ID=dataset1 export FHIR_STORE_ID=fhirstore1 export DICOM_STORE_ID=dicomstore1 export HL7_STORE_ID=hl7v2store1

Create Data Stores

Data in Cloud Healthcare API datasets and stores can be accessed and managed using a REST API that identifies each store using its project, location, dataset, store type and store name. This API implements modality-specific standards for access that are consistent with industry standards for that modality. For example, the Cloud Healthcare DICOM API natively provides operations for reading DICOM studies and series that are consistent with the DICOMweb standard, and supports the DICOM DIMSE C-STORE protocol via an open-source adapter.

Call the API to create a DICOM store:

gcloud beta healthcare dicom-stores create $DICOM_STORE_ID --dataset=$DATASET_ID --location=$REGION

The server returns a path to the newly created store.

Click Check my progress to verify the objective. Create Data Stores

You can also use the curl utility to issue Cloud Healthcare API calls. curl is pre-installed in your Cloud Shell machine. By default, curl does not show HTTP status codes or session-related information; if you would like to see this information please add the "-v" option to all commands in this tutorial.

Try creating a secondary DICOM store by using the below command:

curl -X POST \ -H "Authorization: Bearer "$(sudo gcloud auth print-access-token) \ -H "Content-Type: application/json; charset=utf-8" \ "https://healthcare.googleapis.com/v1beta1/projects/$PROJECT_ID/locations/$REGION/datasets/$DATASET_ID/dicomStores?dicomStoreId=dicomstore2"

Operations that access a modality-specific store use a request path that is comprised of two pieces: a base path, and a modality-specific request path. Administrative operations — which generally operate only on locations, datasets and stores — may only use the base path, but data modality-specific retrieval operations use both the base path (for identifying the store to be accessed) and request path (for identifying the actual data to be retrieved).

Import to DICOM Datasets

In this section you will be importing data from the NIH Chest x-ray set to a DICOM store. For more information on the public dataset, visit the documentation: https://cloud.google.com/healthcare/docs/resources/nih-chest

Call the API to use the import functionality:

gcloud beta healthcare dicom-stores import gcs $DICOM_STORE_ID --dataset=$DATASET_ID --location=$REGION --gcs-uri=gs://gcs-public-data--healthcare-nih-chest-xray/dicom/000000* --project=$PROJECT_ID

Continue to issue this option status request until the result shows done in the response.

Exporting metadata to BigQuery

Run the following to create a dataset in BigQuery:

bq --location=us-east4 mk --dataset --description DICOM $PROJECT_ID:$DATASET_ID

Export the metadata to a new table:

curl -X POST \ -H "Authorization: Bearer "$(gcloud auth print-access-token) \ -H "Content-Type: application/json; charset=utf-8" \ --data "{ 'bigqueryDestination': { 'tableUri': 'bq://$PROJECT_ID.$DATASET_ID.$DICOM_STORE_ID' } }" "https://healthcare.googleapis.com/v1beta1/projects/$PROJECT_ID/locations/$REGION/datasets/$DATASET_ID/dicomStores/$DICOM_STORE_ID:export"

Click Check my progress to verify the objective. Exporting metadata to BigQuery

Exploring the data in BigQuery

In the Console, navigate to Navigation menu > BigQuery.

Under Explorer find your project ID and expand the drop-down.

Find dataset1 and expand the drop-down.

Select dicomstore1 under the drop down and navigate the the Preview tab, where the recent imported metadata is displayed. Scroll right to see the data that was imported into the table.

In the query EDITOR add the following query, then click RUN to find out what patient 25290's findings are:

SELECT StudyDescription FROM dataset1.dicomstore1 where PatientID='88'

See that a Effusions with Cardiomegaly was found.

e70f05da9ccdfcbe.png

Click on COMPOSE NEW QUERY and in the query EDITOR now add the following query to find out who the patients with no findings are:

SELECT PatientID FROM dataset1.dicomstore1 where StudyDescription='No Finding'

Lab review

Cloud Healthcare API provides a comprehensive facility for ingesting, storing, managing, and securely exposing healthcare data in FHIR, DICOM, and HL7 v2 formats. Using Cloud Healthcare API, you can ingest and store data from electronic health records systems (EHRs), radiological information systems (RISs), and custom healthcare applications. You can then immediately make that data available to applications for analysis, machine learning prediction and inference, and consumer access.

Cloud Healthcare API enables application access to healthcare data via widely-accepted, standards-based interfaces such as FHIR STU3 and DICOMweb. These APIs allow data ingestion into modality-specific data stores, which support data retrieval, update, search and other functions using familiar standards-based interfaces.

Further, the API integrates with other capabilities in Google Cloud through two primary mechanisms:

  • Cloud Pub/Sub, which provides near-real-time updates when data is ingested into a Cloud Healthcare API data store, and
  • Import/export APIs, which allow you to integrate Cloud Healthcare API into both Google Cloud Storage and Google BigQuery.

Using Cloud Pub/Sub with Google Cloud Functions enables you to invoke machine learning models on healthcare data, storing the resulting predictions back in Cloud Healthcare API data store. A similar integration with Cloud Dataflow supports transformation and cleansing of healthcare data prior to use by applications.

To support healthcare research, Cloud Healthcare API offers de-identification capabilities for FHIR and DICOM. This feature allows customers to share data with researchers working on new cutting-edge diagnostics and medicines.

Congratulations

In this lab you:

  • Gained a general understanding of Cloud Healthcare API and its role in managing healthcare data.
  • Learned how to create datasets and stores for FHIR and DICOM data.
  • Imported FHIR and DICOM data.

Healthcare_125.png

Finish Your Quest

This self-paced lab is part of the Qwiklabs Cloud Healthcare API Quest. A Quest is a series of related labs that form a learning path. Completing a Quest earns you a badge to recognize your achievement. You can make your badge (or badges) public and link to them in your online resume or social media account. Enroll in this Quest and get immediate completion credit if you've taken this lab. See other available Qwiklabs Quests.

Take your next lab

Continue your quest with Ingesting DICOM Data with the Healthcare API or try one of these suggestions:

End your lab

When you have completed your lab, click End Lab. Qwiklabs removes the resources you’ve used and cleans the account for you.

You will be given an opportunity to rate the lab experience. Select the applicable number of stars, type a comment, and then click Submit.

The number of stars indicates the following:

  • 1 star = Very dissatisfied
  • 2 stars = Dissatisfied
  • 3 stars = Neutral
  • 4 stars = Satisfied
  • 5 stars = Very satisfied

You can close the dialog box if you don't want to provide feedback.

For feedback, suggestions, or corrections, please use the Support tab.

Manual Last Updated: March 10, 2021
Lab Last Tested: March 10, 2021

Copyright 2022 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.