Overview of the Historian ETL Tools

Transferring data from one Historian server to another is typically performed by Proficy Historian collectors. These collectors provide a connected streaming data transfer mechanism (except the Calculation and File collectors). In a system where a steady network connection is not possible or not cost-effective, a periodic file-oriented data transfer is preferred. The Historian Extract, Transform, and Load (ETL) tools consist of a comprehensive set of file-oriented data extraction, transfer, and loading tools.

Potential ways of using ETL tools:
  • Data transfer from any of the following sources:
    • An eDNA server
    • An ODBC data source
    • Proficy Historian
    • PI Historian
  • Data transfer via radio or low bandwidth cellular connection
  • Data transfer where there is no connectivity (read and write using portable media)
  • Data transfer for periodic connectivity applications (for example, ships can transfer data when they arrive at a port)
  • Data migration from OSI PI Server to Proficy Historian
  • Data extraction to import into other applications
  • Data import from other applications
Components of Historian ETL:
  • Extract: Using this tool, you can extract time series data from an eDNA server, an ODBC data source, Proficy Historian or PI Server. For Proficy Historian, you can also extract alarms and events data, perform scaling and absolute deadband compression.
  • Transform: Using this tool, you can transfer data from an onsite Historian server or an ODBC data source to the destination Historian server using a file-sharing application such as FTP, BITS, and so on.
  • Load: Using this tool, you can load data into Proficy Historian. This tool monitors a file directory, unzips the files, and processes them.
Depending on the use case, you can use these tools independently or together.

Recent Enhancements to improve the ETL Load Performance in Historian 2025

  • Improved overall performance by 90 % compared to the previous releases
  • Improved write performance

    The ETL tool can push large data to the Historian servers. For the Historian servers to handle data at that scale and to maximize the write performance of Historian server while loading data from the ETL process, you can increase the Number of Write Threads which allows Historian server to process data in parallel.

    If needed, you can increase the Number of Write Threads from Configuration Hub.

    Configuration Hub

    Before you begin, ensure that you are a member of the iH Security Admins group.

    1. Access Configuration Hub.
    2. In the NAVIGATION section, under the Configuration Hub plugin for Historian, select Systems.

      A list of systems appears in the main section.

    3. Right-click the system whose advanced settings you want to configure, and then select Advanced Configuration.

      The advanced settings of the system appear, displaying the Server section by default.

    4. Enter the value in the Number of Write Threads field.
    5. Select Save.
  • Improved data processing speed using ETL load

    You can create sub folders within the Historian File Path where each folder contains the data for a set of tags.o facilitate this improvement, a new Allow Multiple Folder Processing option is added in HistorianETLLoadConfigTool and MultithreadFileProcessing in the HistorianETLLoad.exe.config file. For more information, refer to Configure the Historian ETL Load tool.

Limitations:
  • The ETL tools do not support array data.