RECOVER

Introduction

The RECOVER project

Background

Details

Use Cases

Technical Information

Programming Languages

Server Information

  • Development -
  • Production -

Database Information

  • Development -
  • Production -

Processes

Running ETL Process

  1. Run the STAR COVID-19 mini-CDM Refresh process per the instructions below (link to come later)

  2. Login to appropriate Unix server (i.e. PuTTY, SSH) that has the Datavant software installed:

    1. DEV: plt-nosql01.dhe.duke.edu

    2. PRD: plp-nosql01.dhe.duke.edu
  3. Change directory to /dwhome/airflow_ci/master/dags/CDRN_ETL/Datavant/
  4. Run the following command:
    1. hash_token_load_master.sh RECOVER

  5. Use a web browser to get to the appropriate Airflow server
    1. DEV: http://plt-nosql01.dhe.duke.edu:8091/admin/

    2. PRD: http://plp-nosql01.dhe.duke.edu:8083/admin/

  6. Run the DAG STAR_Recover

  7. Follow the steps to transfer the file via AWS per the instructions below

    1. STAR RECOVER AWS S3 Info (link to come later)

Refreshing AWS Keys

  1. Follow the steps in the AWS S3 Quick Reference Guide Setup Commands Section for refreshing key values.

Running DeGauss Software

  1. Export data from HSC_PCOR.CVDNEW_PRIVATE_ADDRESS_HISTORY to CSV file.
  2. Transfer CSV file to plt-nosql03.dhe.duke.edu, directory /dwhome/apps/CDRN/RECOVER.
  3. Login to plt-nosql03.dhe.duke.edu
  4. Change directory to /dwhome/apps/CDRN/RECOVER.
  5. Run degauss.sh

SQL Script Migration


Subject AreaSQL Scripts (Links to Git Repository Location)NotesConverted to SQL Server?
1Conditionrecover_condition.sql

2Deathrecover_death.sql

3Death Causerecover_death_cause.sql

4Demographicrecover_demographic.sql

5Diagnosisrecover_diagnosis.sql

6Dispensingrecover_dispensing.sql

7Encounterrecover_encounter.sql

8Enrollmentrecover_enrollment.sql

9Geocodingrecover_geocoding.sql

10Hash Tokenrecover_hash_token.sql

11Immunizationrecover_immunization.sql

12Lab Resultsrecover_lab_result_cm.sql

13LDS Address Historyrecover_lds_address_history.sql

14Medication Administrationrecover_med_admin.sql

15Observation, Clinicalrecover_obs_clin.sql

16Observation, Generalrecover_obs_gen.sql

17Prescribingrecover_prescribing.sql

18PROrecover_pro_cm.sql

19Procedurerecover_procedures.sql

20Providerrecover_provider.sql

21Referralrecover_referral.sql

22Vitalrecover_vital.sql

Troubleshooting / FAQ

References