Deploying Proficy MDC

Deploy Proficy MDC

About this task

The following table outlines the steps that you must perform to deploy and configure Proficy Manufacturing Data Cloud (Proficy MDC).
Note:
You must complete these tasks in the order in which they are listed.
Step Task Purpose
1 Set up Plant Applications SQL Server for replication. To configure the Plant Applications database for replication.
2 On-board the Plant Applications sites to the AWS DMS. To set up the components that are necessary to replicate the data of individual Plant Applications sites and to store the replicated data before ingesting it to Predix Cloud.
3 Setup AWS DMS. To configure the AWS DMS for data replication.
4 Configure AWS site to site VPN To configure connectivity to On Prem database from GE Cloud.
5 Configure AWS DMS. To update the specifications of replication.
Note:
You must perform this step only if you manually on-boarded the sites.
6 Configure DMS to Handle DDL statements. To configure DMS to handle DDL statements.
  • HandleSourceTableAltered – Set this option to true to alter the target table when the source table is altered.
7 Verify the source location of data to be replicated. To confirm that the Plant Applications data that must be replicated to AWS DMS.
8 Verify the target location of replicated data. To confirm that the target location where the replicated data must be stored is accessible to AWS DMS.
9 Create backups of the Plant Applications SQL server transaction log. To schedule a periodic deletion of the Plant Applications SQL server transaction logs so that they do not exceed the allotted system memory.
10 Start the AWS DMS task. The task will perform the following:
  1. Initialize AWS DMS setup for replication.
  2. Full load Increment or, on going replication.
11 Replicate the initial load of files. To replicate the first batch of files in the Plant Applications database.
12 Verify the initial load of files replicated by AWS DMS. To ensure that AWS DMS is correctly replicating the data.

On-board Plant Applications Site

Before you begin

  • Configure a new site in Proficy Manufacturing Data Cloud (Proficy MDC), and keep the following site information handy:
    • Tenant ID
    • Tenant name
    • Enterprise ID
    • Enterprise name
    • Site ID
    • Site name
  • Ensure that replication is enabled for the Plant Applications SQL server instance.

Configure AWS DMS to Handle DDL Statements

About this task

Normally, AWS DMS only handles database DML statements (such as insert, update, and delete). This topic describes how to configure AWS DMS to handle DDL statements as well by using the HandleSourceTableAltered action. AWS DMS can then handle DDL statements such as create table, drop table, alter table, add column, drop column, and so on.
  • HandleSourceTableAltered action – Set this option to true to alter the target table when the source table is altered.
  • Following are the steps to alter the target table using HandleSourceTableAltered action:
    • Access the AWS DMS console.
    • Select the replication task, then select Actions and then Stop.
    • Select the replication task again, then select Actions and then Modify.
    • Under task settings, switch to the JSON editor.
    • Find ChangeProcessingDdlHandlingPolicy.
    • Add or update the HandleSourceTableAltered action to True.
    • Save the task and resume the task.

Verify Source Location of Data

About this task

You must verify whether the Plant Applications data to be ingested to Predix Cloud is accessible to AWS DMS for replication.
Important:
In this task, angle brackets (< >) indicate placeholder text. If such placeholder text is part of a command, you must replace it with an appropriate value before running the command.

Procedure

  1. Access the AWS DMS console.
  2. Navigate to Source Endpoints and create a new source endpoint.
  3. In the Connection section:
    • Endpoint Type: Source endpoint
    • Under Endpoint Configuration, provide a name for the source endpoint at Endpoint identifier.
    • Select Source Engine as Microsoft SQL Server.
    • Provide details like server IP, SQL port, and credentials.
    • Test the connection, and create the endpoint.
    Note:
    The user for AWS DMS must use SQL authentication only. Windows and Domain authentication cannot be used for AWS DMS.
    AWS DMS pings the source location of data to verify whether it is accessible.
    Note:
    If the connection test fails, verify that you have specified the correct IP addresses and login credentials.

Verify Target Location of Data

About this task

You must verify that the S3 path where the Plant Applications data must be replicated is accessible to AWS DMS.
Important:
In this task, angle brackets (< >) indicate placeholder text. If such placeholder text is part of a command, you must replace it with an appropriate value before running the command.

Procedure

  1. Access the AWS DMS console.
  2. In the endpoint option, provide the following details for the Target Endpoint (that is, S3 Bucket location):
    • Endpoint Type: Target endpoint
    • Under Endpoint Configuration, provide a name for the target endpoint at Endpoint identifier.
    • Select Target Engine as Amazon S3.
    • Enter ARN value, Bucket Name, and Folder Location (tenant_id=<TENANT_VAL>/site_id=<SITE_VAL>) in the corresponding fields.
    • Under Endpoint Settings, add the following new settings:
      • DataFormat: parquet
      • CompressionType: GZIP
      • CdcMaxBatchInterval: 900
    • Once the details are verified, click on Create Endpoint.
  3. Select Test Connection.
    AWS DMS pings the source location of data to verify whether it is accessible.
    Note:
    If the connection test fails, verify that you have specified the correct directory in the Directory box.

Replicate Initial Load to AWS DMS

About this task

The data replication starts with the replication of files that are currently stored in the Plant Applications database. The first batch of files to be replicated is considered as the initial load. After the initial load is replicated, the modifications made to the data in the Plant Applications database are replicated as incremental loads.

Procedure

  1. Access the AWS DMS console.
  2. In the AWS DMS task, create a task with the following details:
    • Select the source and target endpoints created previously.
    • Select the tables to be replicated.
    • Select Migration type as Migrate existing data.
    • Select Migration task startup configuration as Manually later.
    • Once details are verified, create the task.
  3. Navigate to the Database migration task and start the task.
  4. After the task completes, verify the data in the target Amazon S3 path.