Data Transformation Manager (DTM)

The integration service starts a DTM process to run each session and command task with in a workflow. The DTM process performs session validations, creates threads to initialize the session, read, write,and transform data and handles pre and post session operations. 
When the workflow reaches a session , the integration service process starts the DTM process. The DTM is the process associated with the session task. The DTM process performs the following tasks.
  • DTM creates a session log.
  • Verifies connection object permissions.
  • Retrieves and validates session information from repository.
  • DTM performs pushdown optimization.
  • Runs pre- and post session shell commands, stored procedures and SQL.
  • Add partitions to the session when the session is configured for dynamic partitioning.
  • Validate source and target code pages.
  • Expand the service process variables, session parameters, and mapping variables and parameters.
  • Sends a request to start worker DTM processes on other nodes when the session is configured to run on a grid.
  • Sends post session email.
  • Creates and runs mapping, reader, writer and transformation threads to extract, transform and load data.  
Thread Types
The DTM uses multiple threads to process data in a session. The main DTM thread is called the master thread. The master thread creates different types of threads for a session. The type of threads the master thread creates depend on the pre and post session properties as well as the types of transformations in the mapping.
The master thread can create the following types of threads
  • Mapping threads
  • Pre- and post- session Threads
  • Reader Threads
  • Transformation Threads
  • Writer Threads
DTM Buffer Memory
  • The DTM allocates process memory for the session and divide it into buffers. This is known as buffer memory.   
  • The integration service process launches the DTM. The DTM allocates buffer memory to the session based on the DTM buffer size setting in the session properties. BY default the integration service determines the size of the buffer memory. However you may want to configure the buffer memory and buffer block size manually. The default memory allocation is 12,000,000 bytes.
  • The DTM divides the memory into buffer blocks as configured in the buffer block size setting in the session properties. The reader , transformation and writer threads use buffer blocks to move data from source to targets.                           
  • We can improve session performance by increasing buffer memory when we run a session handling large volume of character data and the integration service runs in Unicode mode. In Unicode mode, the integration service uses the double bytes to move characters, so increasing buffer memory might improve session performance.
  • If the DTM cannot allocate the configured amount of buffer memory for the session, the session cannot initialize. Informatica recommends you to allocate no more than 1GB for DTM buffer memory.    
Powered by Blogger.