1. Talend 5.4 Architecture
Adam Pemble
Talend Professional Services

2. Talend Unified Platform
DSC /
(includes DSC)
© Talend 2013

3. Talend Product Options - Platform
Talend Platform:
Universal
for Big Data
for Data Management
for Data Services
for Data Services with Big Data
for Enterprise Integration
for Enterprise Integration with Big Data
for MDM
for MDM with Big Data
© Talend 2013

4. Talend Product Options - Enterprise
Talend Enterprise:
Big Data
Data Integration
ESB
© Talend 2013

5. Universal Architecture
DSC /
(includes DSC)
© Talend 2013

6. DI / DQ Architecture Data Stewardship Console Data StewardshipDB
Data Stewardship
Console
Data Stewardship
Console Web UI
© Talend 2013

7. Data Services ArchitectureDB
Data Stewardship
Console
Data Stewardship
Console Web UI
© Talend 2013

8. Data Services with BPM ArchitectureDB
Data Stewardship
Console
Data Stewardship
Console Web UI
© Talend 2013

9. ESB Architecture
© Talend 2013

10. MDM / ESB Architecture
DSC /
(includes DSC)
© Talend 2013

11. Talend Administration Center (TAC)
Purpose: Management of the Talend Environment:
Environment configuration
User / role administration
Project administration
Job scheduling and execution (Job Conductor)
Monitoring
Authorisation
© Talend 2013

12. Talend Administration Center (TAC)

13. Talend Administration Center (TAC)
Technical details:
Server component
Web application – accessed via browser
One per Talend environment
Hosted on Tomcat, JBoss or Weblogic
Talend Installer can install TAC and Tomcat (v6) together
Can contain a number of child web applications: AMC, JBoss Drools Guvnor, Kibanna
The web application server is typically run as a Service / Daemon
© Talend 2013

14. TAC Database Purpose: Technical details: Server component
Database to hold details about the Talend environment (users, configuration, scheduled tasks etc.)
Accessed by TAC
Technical details:
Various relational databases supported
Small database – does not hold a large volume of data
© Talend 2013

15. Talend Studio Purpose: Single Studio for all Talend Development tasks
Different perspectives for different types of development, e.g. ‘Data Integration’, ‘MDM’ etc.
Can execute Jobs / DQ reports etc.
© Talend 2013

16. Talend Studio
© Talend 2013

17. Talend Studio Technical details: Client component
Based on Eclipse:
Thick client: one per developer
© Talend 2013

18. SVN Repository Purpose: Store for Talend Projects
Contents of projects e.g. Jobs, connections, schema definitions etc. are stored in SVN in Talend metadata format (i.e. properties files, XML files etc.) Does not contain binaries of these Talend objects
Shared libraries e.g. JDBC drivers are also stored in SVN
Provides a versioning system for Talend objects / projects
© Talend 2013

19. SVN Repository
© Talend 2013

20. SVN Repository Technical details: Server Component
Apache Subversion (often abbreviated SVN, after the command ‘svn’) is a software versioning and revision control system distributed as free software under the Apache License.
Can be installed in some circumstances by the Talend Installer, but this is not common
Other open source tools build on SVN to provide easy to use management interfaces e.g. Visual SVN Server
© Talend 2013

21. SVN Repository Technical details (cont):
By default Talend treats SVN as a ‘black box’ – you should not normally have to interact directly with SVN during day to day development
SVN is typically run as a Service / Daemon, often in conjunction with Apache for http access
Typically you will have one SVN server per Talend environment, but many configuration options are available.
© Talend 2013

22. Talend CmdLine (CommandLine)
Purpose:
Performs operations on the Talend repository
Most important function is to extract Job metadata from the repository and compile it into executable code when required to do so by the TAC (Job Conductor)
Can also be used in a number of different interactive and scripted modes which allow for operation automation (particularly deployment between Talend environments) and mass updates
© Talend 2013

23. Talend CmdLine (CommandLine)

24. Talend CmdLine (CommandLine)
Technical details:
Server Component
Is actually a copy of the Studio running in non-GUI mode
When Studio patches are applied, they should also be applied to the CommandLine
Is typically run as a Service / Daemon
Normally one CommandLine per Talend environment
© Talend 2013

25. Talend Job Server Purpose: Executes Talend Jobs
Used by the Job Conductor within TAC to execute Jobs
Can be used by the Talend Studio to execute jobs ‘server side’
© Talend 2013

26. Talend Job Server
© Talend 2013

27. Talend Job Server Technical details: Server Component
Is typically run as a Service / Daemon
No license restrictions on the number of Job Servers that can be installed
This allows placement for maximum performance
Various options for resiliency (covered later in this presentation)
Can be secured if necessary
© Talend 2013

28. Talend Data Quality Portal
Purpose:
Allow business users to view Data Quality reports and dashboards via a Web interface
© Talend 2013

29. Talend Data Quality Portal
Technical details:
Server component
Web application – accessed via browser
Can be many instances in a single Talend environment
Hosted on Tomcat only (can be same Tomcat as TAC)
Talend Installer can install TDQP and Tomcat (v6) together
As well as the relational DB for the report data, a HSQL database is also used for environment / user management
The web application server is typically run as a Service / Daemon
© Talend 2013

30. Talend DQ Data Mart Purpose:
Database that holds the results of the execution of DQ reports
Written to by running a report in the Studio and / or a DI Job that executes a specified DQ report
© Talend 2013

31. Talend DQ Data Mart Technical details: Server component
Number of relational databases supported, subset of those supported by rest of Talend platform
Can be many instances in a single Talend environment
Small – Medium size database: does not hold any actual source data:
Evolutionary Reports: all results from all report runs
Basic reports: last run of report
© Talend 2013

32. Talend Data Stewardship Console
Purpose:
Provides a Web UI for Data Stewards to resolve issues with records and possible matches
Part of MDM or Standalone
Used by MDM Integrated Matching
© Talend 2013

33. Talend Data Stewardship Console
Technical details:
Server component
Web application – accessed via browser
Can be many instances in a single Talend environment
Hosted on Tomcat or JBoss
If installed with MDM uses MDM authentication system and is a child application of MDM Web UI
If Standalone, basic user management via a file
Talend Installer can install DSC with TAC or MDM
The web application server is typically run as a Service / Daemon
© Talend 2013

34. Talend DSC Database Purpose: Database that holds the DSC tasks

35. Talend DSC Database Technical details: Server component
Number of relational databases supported, subset of those supported by rest of Talend platform
Can be many instances in a single Talend environment
Small – Medium size database: depends on the number of DSC tasks held in the DB at any given time
© Talend 2013

36. Talend MDM Server Purpose: Master Data Management Server
Web UI for MDM Governance tasks
MDM Engine
Master Data Indexing
Configuration of MDM Storage layer
Integrated workflow
© Talend 2013

37. Talend MDM Server
© Talend 2013

38. Talend MDM Server Technical details: Server component
Can be many instances in a single Talend environment, more typically one per environment – runtime licence dependant
Runs in a Talend modified version of JBoss (should only be used for Talend MDM)
JBoss / MDM are installed by the Talend Installer
BPM can run in the same or a different runtime (Talend Platform Universal only)
The MDM (JBoss) server is typically run as a Service / Daemon
© Talend 2013

39. Talend MDM Server Storage
Purpose:
Databases / Schemas to hold:
Master Data
Staging Data
MDM Journal
MDM System data
Etc.
© Talend 2013

40. Talend MDM Server Storage
Technical details:
Server component
Number of relational databases supported
One set of Databases / Schemas required per MDM server
Database can be very large, depending on the volume of Master Data records
© Talend 2013

41. Q&A