> For the complete documentation index, see [llms.txt](https://developers.mpdx.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developers.mpdx.org/staging/org_data_transfer.md).

# org\_data\_transfer

## Building an export:

1. turn on extract database:

   `assume-role -e p -- aws rds start-db-instance --db-instance-identifier mpdx-api-extract-prod`
2. reset transfer database

   ```
    run-locally -e s \
      --docker-args "\
        -e DATADOG_TRACE=false\
        -e DB_ENV_POSTGRESQL_DB=mpdx_extract\
        -e DB_ENV_POSTGRESQL_USER=mpdx_extract_admin\
        -e DB_ENV_POSTGRESQL_PASS=REDACTED\
        -e DB_PORT_5432_TCP_ADDR=mpdx-api-extract-prod.ctzggtk79wff.us-east-1.rds.amazonaws.com" \
      -p false bash
   ```
3. start data transfer:
   * `prod_console`
   * `Organization::Restorer.restore("05df083a-6836-4148-a15a-ae6d72d00749")`
4. wait for sidekiq batch to finish
5. Dump transfer database to compressed file:

   ```
   pg_dump -Fc \
      -h mpdx-api-extract-prod.ctzggtk79wff.us-east-1.rds.amazonaws.com \
      -f ./transfer-test1.dmp \
      -U mpdx_extract_admin \
      -d mpdx_extract
   ```
6. Transfer all s3 files to transfer bucket:

   `Organization::Restorer.new("05df083a-6836-4148-a15a-ae6d72d00749").restore_attachments(progress_bar: true)`
7. Sync all files from transfer bucket to a local directory:

   `assume-role -e p -- aws s3 sync s3://mpdx-transmit-production docker_data/s3`
8. Zip dir:

   `tar -cf - docker_data/s3 | xz -1 -c - -> docker_data/s3.tar.xz`
9. turn off extract database:

   `assume-role -e p -- aws rds stop-db-instance --db-instance-identifier mpdx-api-extract-prod`

## Run MPDX locally with export

1. pg restore:
   * `git checkout public-dockerfile`
   * Start the api database server: `docker-compose up -d db`
   * Create database: `createdb -h localhost -p 54322 -U postgres mpdx_extract`
   * Restore data to database:

     `pg_restore -c -d mpdx_extract -h localhost -p 54322 -U postgres --no-privileges --no-owner ./transfer-test.dmp`
2. build mpdx\_api docker image:
   * `bin/public_build.sh` (requires SIDEKIQ\_CREDS to be set on host env)
3. run mpdx api:
   * `docker-compose up`
4. run mpdx\_web:
   * `cd mpdx_web`
   * `npm install`
   * `API_URL=http://localhost:50000/api/v2 OAUTH_URL=http://auth.lvh.me:50000/auth/user npm run start`
5. open <http://localhost:8081>