Versions Compared

Old Version 6

changes.mady.by.user Daniel Popham

Saved on

compared with

New Version 7

changes.mady.by.user Daniel Popham

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This page will be used to track the different projects that need to be moved from the Oracle Exadata environment to another environment.

Database Schema List

The list of affected projects are below:

  • ACT - Decommissioned process as of May 2023. However, there is data in the following schemas that should be archived somewhere before Exadata is decommissioned:

    • ACT_I2B2_DATA

    • ACT_I2B2_HIVE

    • ACT_I2B2_IMDATA

    • ACT_I2B2_METADATA

    • ACT_I2B2_PM

    • ACT_I2B2_SHRINE_ONT

    • ACT_I2B2_WORKDATA

    • NOTE: There are some other schemas starting with ACT_TEST that can be deleted as they are no longer being used for ACT testing and development.
  • CODI - There is 1 schema that currently contains data used for the CODI project, supported by the PCORNet CDM data.
    • CODI
  • CC (Carolina Collaborative) - Old project that utilized the same CDM structure as ACT. Decommissioned before 2020. However, required to keep data until 2025 for archival/historical purposes. Data in the following schemas should be archived somewhere before Exadata is decommissioned: 

    • HSC_I2B2_DATA

    • HSC_I2B2_HIVE

    • HSC_I2B2_IMDATA

    • HSC_I2B2_METADATA

    • HSC_I2B2_PM

    • HSC_I2B2_SHRINE_ONT

    • HSC_I2B2_WORKDATA

  • STAR - Project that contains the PCORNet CDM data that is refreshed quarterly. Only 1 schema is involved with this project:
    • HSC_PCOR
  • RXNorm Refresh - Project related to STAR that refreshes RxNorm data on a quarterly basis, before the STAR quarterly refresh. Following schemas are used
    • RXNORM
    • RXNORM_2017 - Actually used for N3C project. These tables should be moved somewhere else that makes more sense.
  • Bayer - Data extract from external database. Not affected by migration.

Airflow Process List

  • CDRNLabReload    
  • CDRNStaticIdDAG    
  • CODI_DAG    
  • CVD19GetPatKeys
  • STAR_CopyPcornetTables
  • STAR_ETL_Master    
  • STAR_ETL_PostOp    
  • STAR_Inc_Load    
  • STAR_N3C_Hash_Tokens    
  • STAR_Recover    


Exadata Schema NameProject NameProcess TypeProcess NameProcess LocationOwnerNotes

ACT_I2B2_DATA

ACTN/AN/AN/ADaniel Popham




Decommissioned project as of May 2023. However, there is data in the following schemas that should be archived somewhere before Exadata is decommissioned.







ACT_I2B2_HIVE

ACTN/AN/AN/ADaniel Popham

ACT_I2B2_IMDATA

ACTN/AN/AN/ADaniel Popham

ACT_I2B2_METADATA

ACTN/AN/AN/ADaniel Popham

ACT_I2B2_PM

ACTN/AN/AN/ADaniel Popham

ACT_I2B2_SHRINE_ONT

ACTN/AN/AN/ADaniel Popham

ACT_I2B2_WORKDATA

ACTN/AN/AN/ADaniel Popham
ACT_TEST_DATAACTN/AN/AN/ADaniel Popham




Decommissioned project as of May 2023. All of these schemas can be removed.







ACT_TEST_HIVEACTN/AN/AN/ADaniel Popham
ACT_TEST_IMDATAACTN/AN/AN/ADaniel Popham
ACT_TEST_METADATAACTN/AN/AN/ADaniel Popham
ACT_TEST_PMACTN/AN/AN/ADaniel Popham
ACT_TEST_SHRINE_ONTACTN/AN/AN/ADaniel Popham
ACT_TEST_WORKDATAACTN/AN/AN/ADaniel Popham
CODICODIAirflow DAG

CODI_DAG

DEV: plt-nosql01.dhe.duke.edu

PRD: plp-nosql01.dhe.duke.edu

Daniel PophamContains data used for the CODI project, supported by the PCORNet CDM data.

HSC_I2B2_DATA

Carolinas Collaborative (CC)N/AN/AN/ADaniel Popham







Decommissioned project some time before 2020. However, required to keep data until end of year 2025 for archival/historical purposes. Data in the following schemas should be archived somewhere before Exadata is decommissioned







HSC_I2B2_HIVECarolinas Collaborative (CC)N/AN/AN/ADaniel Popham
HSC_I2B2_IMDATACarolinas Collaborative (CC)N/AN/AN/ADaniel Popham
HSC_I2B2_METADATACarolinas Collaborative (CC)N/AN/AN/ADaniel Popham
HSC_I2B2_PMCarolinas Collaborative (CC)N/AN/AN/ADaniel Popham
HSC_I2B2_SHRINE_ONTCarolinas Collaborative (CC)N/AN/AN/ADaniel Popham
HSC_I2B2_WORKDATACarolinas Collaborative (CC)N/AN/AN/ADaniel Popham





HSC_PCOR

N3C






Airflow DAG

STAR_N3C_Hash_Tokens    





DEV: plt-nosql01.dhe.duke.edu

PRD: plp-nosql01.dhe.duke.edu


Daniel PophamProject that runs monthly to generate data for N3C project from monthly refresh.
PCORNet/STAR
Airflow DAGs
  • CDRNLabReload    
  • CDRNStaticIdDAG    
  • CVD19GetPatKeys
  • STAR_CopyPcornetTables
  • STAR_ETL_Master    
  • STAR_ETL_PostOp    
  • STAR_Inc_Load
   
  • STAR_N3C_Hash_Tokens    
    • STAR_Recover
    •    

    DEV: plt-nosql01.dhe.duke.edu

    PRD: plp-nosql01.dhe.duke.edu

    Daniel Popham

    Project that contains the PCORNet CDM data that is refreshed both quarterly and monthly.

    RECOVER

    STAR_Recover    

    		Daniel Popham

    Project that runs quarterly to generate data for RECOVER project based on available PCORNet refresh data .

    BayerPython Script\\duhs-clin-nc1\duhs_pmn_dmc_fs\Bayer\BayerExtract.py

    vwp-pmndmc.dhe.duke.edu

    		Daniel Popham

    Data extract from external database

    . Not affected by migration

    into HSC_PCOR schema.

    RXNORMPCORNet/STARUnix Shell Script/dwhome/airflow_ci/master/dags/CDRN_ETL/RxNormLoad/RxNormLoad.sh

    DEV: plt-nosql01.dhe.duke.edu

    PRD: plp-nosql01.dhe.duke.edu

    		Daniel PophamProcess related to the PCORNet/STAR project that refreshes RxNorm data on a quarterly basis, before the STAR quarterly refresh. Manually run.
    RXNORM_2017PCORNet/STARWindows Batch File\\duhs-clin-nc1\duhs_pmn_dmc_fs\N3C\Project\N3C_Main.batvwp-pmndmc.dhe.duke.eduDaniel PophamActually used for N3C project. These tables should be moved somewhere else that makes more sense.