1Managed by UT-Battelle for the U.S. Department of Energy TriBITS TriBITS Foundations and Updates Roscoe A. Bartlett, Ph.D.

Slides:



Advertisements
Similar presentations
•7/12 /07 F-1 © 2010 T. Horton CS 4240 Principles of SW Design Packages in Java and UML.
Advertisements

Simple Git Steve Pieper. Topics Git considerations and Slicer Git as if it were svn Git the way it is meant to be.
Tribal Build, Integrate, and Test System Roscoe A. Bartlett Computational Engineering and Energy Sciences Group, Oak Ridge National.
VisIt Software Engineering Infrastructure and Release Process LLNL-PRES Lawrence Livermore National Laboratory, P. O. Box 808, Livermore,
Version Control Systems Phil Pratt-Szeliga Fall 2010.
Patterns & practices Symposium 2013 Introducing Git version control into your team Mark
Version Control with git. Version Control Version control is a system that records changes to a file or set of files over time so that you can recall.
1 Introduction to Tool chains. 2 Tool chain for the Sitara Family (but it is true for other ARM based devices as well) A tool chain is a collection of.
Git for Version Control These slides are heavily based on slides created by Ruth Anderson for CSE 390a. Thanks, Ruth! images taken from
Trilinos Coding and Documentation Guidelines Roscoe A. Bartlett Trilinos Software Engineering Technologies and Integration Lead Computer Science and Mathematics.
Page 1 Multi-Repository Development and Integration in CASL using TriBITS Roscoe A. Bartlett Oak Ridge National Laboratories Trilinos User Group Meeting.
SCRAM Software Configuration, Release And Management Background SCRAM has been developed to enable large, geographically dispersed and autonomous groups.
Overview of Git Workflows for CSE Software Trilinos Spring Developers Meeting May 13, 2015 Roscoe A. Bartlett Oak Ridge National Lab Computational Eng.
Page 1 Trilinos Software Engineering Technologies and Integration Capability Area Overview Roscoe A. Bartlett Trilinos Software Engineering Technologies.
Page 1 Trilinos Software Engineering Technologies and Integration Capability Area Overview Roscoe A. Bartlett Department.
Unit tests, Integration tests Physics tests Andrea Dotti, Gunter Folger, Pere Mato CERN – PH/SFT Geant4 workshop 2012.
SKA/KAT SPIN Presentation Software Engineering (!?) Robert Crida.
Version Control with Subversion Quick Reference of Subversion.
Page 1 Trilinos Usage of CMake/CTest/CDash Roscoe A. Bartlett Department of Optimization & Uncertainty Estimation Sandia.
Page 1 Trilinos Software Engineering Status and Future Issues Roscoe A. Bartlett Department of Optimization & Uncertainty.
The Pipeline Processing Framework LSST Applications Meeting IPAC Feb. 19, 2008 Raymond Plante National Center for Supercomputing Applications.
Page 1 Trilinos Software Engineering Technologies and Integration Capability Area Overview Roscoe A. Bartlett Department.
Git Workflow Building Blocks for Improved VERA Development, Integration, and Deployments Roscoe A. Bartlett Physics Integration Infrastructure Team Lead.
Git – versioning and managing your software L. Grewe.
GIT An introduction to GIT Source Control. What is GIT (1 of 2) ▪ “Git is a free and open source distributed version control system designed to handle.
Branching. Version Control - Branching A way to write code without affecting the rest of your team Merge branches to integrate your changes.
Page 1 Trilinos Release Improvement Issues Roscoe A. Bartlett Department of Optimization & Uncertainty Estimation Trilinos.
Version control Using Git Version control, using Git1.
Page 1 Multi-Repository Development and Integration with TriBITS Roscoe A. Bartlett Oak Ridge National Laboratories Trilinos User Group Meeting October.
…using Git/Tortoise Git
SWEN 302: AGILE METHODS Roma Klapaukh & Alex Potanin.
Object-Oriented Analysis & Design Subversion. Contents  Configuration management  The repository  Versioning  Tags  Branches  Subversion 2.
CMake For Trilinos Developers
Computer Science and Engineering The Ohio State University  Widely used, especially in the opensource community, to track all changes to a project and.
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear.
4/1/05F-1 © 2001 T. Horton CS 494 Object-Oriented Analysis & Design Packages and Components in Java and UML.
Generalization and externalization of the Trilinos package- based system Roscoe A. Bartlett Trilinos Software Engineering Technologies and Integration.
Version Control Systems. Version Control Manage changes to software code – Preserve history – Facilitate multiple users / versions.
Trilinos Framework Update Roscoe A. Bartlett James M. Willenbring Michael N. Phenow Trilinos User Group Monday, October 31st, : :30 am Building.
Page 1 Almost Continuous Integration for the Co-Development of Highly Integrated Applications and Third Party Libraries Roscoe A. Bartlett
1Managed by UT-Battelle for the U.S. Department of Energy TriBITS Software Engineering Processes for the CASL Innovation Hub Roscoe A. Bartlett, Ph.D.
CPSC 871 John D. McGregor Change management Module 2 Session 3.
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. Testing Spring Applications Unit Testing.
G.Govi CERN/IT-DB 1 September 26, 2003 POOL Integration, Testing and Release Procedure Integration  Packages structure  External dependencies  Configuration.
Page 1 CMake Trilinos? Roscoe A. Bartlett Department of Optimization & Uncertainty Estimation Esteban J. Guillen Department.
Version Control and SVN ECE 297. Why Do We Need Version Control?
Introduction to Git Yonglei Tao GVSU. Version Control Systems  Also known as Source Code Management systems  Increase your productivity by allowing.
SECTION 1: CODE REASONING + VERSION CONTROL slides borrowed and adapted from Alex Mariakis and CSE 390a Justin Bare & Deric Pang.
1Managed by UT-Battelle for the U.S. Department of Energy Roadmap for Sustainable CSE Ecosystems A Roadmap for Sustainable Ecosystems of CSE Software Roscoe.
TEAM FOUNDATION VERSION CONTROL AN OVERVIEW AND WALKTHROUGH By: Michael Mallar.
Process changes: Internal processes of CASA, external contributions, release schedule Mark G. Rawlings, CASA Build & Test Lead NRAO, Charlottesville Acknowledgements:
Information Systems and Network Engineering Laboratory I DR. KEN COSH WEEK 1.
1 Ivan Marsic Rutgers University LECTURE 2: Software Configuration Management.
Principles of Package Design COMPOSE AT A HIGHER LEVEL OF ABSTRACTION.
GIT Version control. Version Control Sharing code via a centralized DB Also provides for Backtracking (going back to a previous version of code), Branching.
Build and Test system for FairRoot
4 Version control (part 1)
Open-O Integration Project Introduction
I Don’t Git It: A beginner’s guide to git Presented by Mathew Robinson
11 Version control (part 2)
LECTURE 2: Software Configuration Management
Releases and developments
Akshay Narayan git up to speed with RCS Akshay Narayan
LECTURE 3: Software Configuration Management
Getting Started with Git and Bitbucket
P Almost Continuous Integration for the Co-Development of Highly Integrated Applications and Third Party Libraries Roscoe A. Bartlett
ABHISHEK SHARMA ARVIND SRINIVASA BABU HEMANT PRASAD 08-OCT-2018
Patrick Cozzi University of Pennsylvania CIS Fall 2012
Version Control with Git and GitHub
Overview Activities from additional UP disciplines are needed to bring a system into being Implementation Testing Deployment Configuration and change management.
Presentation transcript:

1Managed by UT-Battelle for the U.S. Department of Energy TriBITS TriBITS Foundations and Updates Roscoe A. Bartlett, Ph.D. Computational Engineering and Energy Sciences Group, Oak Ridge National Laboratory Trilinos Developers Meeting October 29, 2015

2Managed by UT-Battelle for the U.S. Department of Energy TriBITS What is TriBITS? Framework for large, distributed multi-repository CMake projects Reduce boiler-plate CMake code and enforce consistency across large distributed projects Subproject dependencies and namespacing architecture (packages) Automatic package dependency handling (directed acyclic graph) Additional functionality missing in raw CMake Change default CMake behavior when necessary Additional tools for agile software development processes (e.g. Continuous Integration (CI)) History of TriBITS: 2007: Initially developed as a CMake package architecture for Trilinos 2011: Generalized and extended for CASL VERA 2014: Source code hosted on GitHub

3Managed by UT-Battelle for the U.S. Department of Energy TriBITS Raw CMake vs. TriBITS

4Managed by UT-Battelle for the U.S. Department of Energy TriBITS Raw CMakeLists.txt File # Build and install library SET(HEADERS hello_world_lib.hpp) SET(SOURCES hello_world_lib.cpp) ADD_LIBRARY(hello_world_lib ${SOURCES}) INSTALL(TARGETS hello_world_lib DESTINATION lib) INSTALL(FILES ${HEADERS} DESTINATION include) # Build and install user executable ADD_EXECUTABLE(hello_world hello_world_main.cpp) TARGET_LINK_LIBRARIES(hello_world hello_world_lib) INSTALL(TARGETS hello_world DESTINATION bin) # Test the executable ADD_TEST(test ${CMAKE_CURRENT_BINARY_DIR}/hello_world) SET_TESTS_PROPERTIES(test PROPERTIES PASS_REGULAR_EXPRESSION "Hello World") # Build and run some unit tests ADD_EXECUTABLE(unit_tests hello_world_unit_tests.cpp) TARGET_LINK_LIBRARIES(unit_tests hello_world_lib) ADD_TEST(unit_test ${CMAKE_CURRENT_BINARY_DIR}/unit_tests) SET_TESTS_PROPERTIES(unit_test PROPERTIES PASS_REGULAR_EXPRESSION "All unit tests passed")

5Managed by UT-Battelle for the U.S. Department of Energy TriBITS TriBITS Package CMakeList.txt File Avoid duplication and reduce boiler-plate commands Install libs & headers but not (test) execs by default (most common use case) Library linking automatically handled by default (between Packages, execs/libs) Automatic namespacing of test & exec names Allows for all library names to be prefixed (needed for Linux distributions) Consistent handling of test enables/disables based on various criteria TRIBITS_PACKAGE(HelloWorld) TRIBITS_ADD_LIBRARY(hello_world_lib HEADERS hello_world_lib.hpp SOURCES hello_world_lib.cpp) TRIBITS_ADD_EXECUTABLE(hello_world NOEXEPREFIX SOURCES hello_world_main.cpp INSTALLABLE) TRIBITS_ADD_TEST(hello_world NOEXEPREFIX PASS_REGULAR_EXPRESSION "Hello World") TRIBITS_ADD_EXECUTABLE_AND_TEST(unit_tests SOURCES hello_world_unit_tests.cpp PASS_REGULAR_EXPRESSION "All unit tests passed") TRIBITS_PACKAGE_POSTPROCESS()

6Managed by UT-Battelle for the U.S. Department of Energy TriBITS TriBITS Structural Elements

7Managed by UT-Battelle for the U.S. Department of Energy TriBITS Packages + Subpackages = Software Engineering (SE) Packages TriBITS Structural Units TriBITS Project: Complete CMake “Project” Overall projects settings TriBITS Repository: Collection of Packages and TPLs Unit of distribution and integration Typically a version control (git) repository TriBITS Package: Encapsulated collection of related software & tests Unit of testing, namespacing, documentation, and reuse Lists dependencies on SE Packages & TPLs TriBITS Subpackage: Optional partitioning of package software & tests Primarily for dependency management (SE principles) TriBITS TPLs (Third Party Libraries): Specification of external dependencies (libs) Required or optional dependency Single definition across all packages Can use native CMake Find.cmake modules See:

8Managed by UT-Battelle for the U.S. Department of Energy TriBITS VERA Meta-Project, Repositories, Packages & Subpackages VERA Trilinos Epetra … Teuchos Core Comm ParameterList … NOX SCALE/Exnihilo Nemesis … Insilico Neutronics … Shift MPACT VERAInExt VERAIn … VERA meta-project: Git repository and TriBITS meta-project (contains no packages) TriBITS repos:: Trilinos, VERAInExt, COBRA-TF, MPACT, SCALE … TriBITS packages: Teuchos, Epetra, VERAIn, Insilico, COBRA_TF, MPACT_Drivers, … TriBITS subpackages: TeuchosCore, InsilicoNeutronics … TriBITS SE packages: TeuchosCore, Teuchos, VERAIn, Insilico, InsilicNeutronics, … COBRA-TF COBRA_TF MPACT_libs MPACT_Drivers … …

9Managed by UT-Battelle for the U.S. Department of Energy TriBITS Flexibility in TriBITS Projects and Repositories MPACT Trilinos SCALE VERAInExt COBRA-TF MPACT The same TriBITS repositories can be arranged into multiple TriBITS CMake projects! SCALE (Exnihilo) Trilinos SCALE VERAInExt Exnihilo COBRA-TF Trilinos

10Managed by UT-Battelle for the U.S. Department of Energy TriBITS TriBITS and VC Repos for CASL VERA Trilinos (SNL) TeuchosWrappersExt (Multi Inst.) VERAInExt (Multi Inst.) COBRA-TF (PennState) MPACT (U.Mich.) SCALE (ORNL) VUQDemos (SNL) MOOSEExt MOOSE / Bison (INL) DatraTransferKit (ORNL) Exnihilo (ORNL) DakotaExt Dakota (SNL) PSSDriversExt (Multi Inst.) Outer boxes are all TriBITS and VC repos (all native git or snapshotted to git) Inner boxes (e.g. Exnihilo, MOOSE, Dakota) are VC repos but not TriBITS repos MAMBA (LANL)

11Managed by UT-Battelle for the U.S. Department of Energy TriBITS VERA/cmake/ExtraRepositoriesList.cmake TRIBITS_PROJECT_DEFINE_EXTRA_REPOSITORIES( TriBITS "" GIT "" ${TriBITS_REPO_TYPE} Trilinos "" GIT "" Continuous TeuchosWrappersExt "" GIT "" Continuous MAMBA "" GIT "" Continuous COBRA-TF "" GIT "" Continuous VERAInExt "" GIT "" Continuous VERAData "" GIT NOPACKAGES ${VERADATA_CAT} DataTransferKit "" GIT "" Continuous MOOSEExt "" GIT "" Continuous MOOSE MOOSEExt/MOOSE GIT NOPACKAGES Continuous SCALE "" GIT "" Continuous Exnihilo SCALE/Exnihilo GIT NOPACKAGES Continuous MPACT "" GIT "" Continuous LIMEExt "" GIT "" Nightly PSSDriversExt "" GIT "" Continuous DakotaExt "" GIT Continuous Dakota DakotaExt/Dakota GIT NOPACKAGES Continuous VUQDemos "" GIT "" Nightly ) Official version of VERA in on master branch used for CI & Nightly testing Partial set of repos can be cloned (protected by different groups) Non-git repos are converted into git repos: Dakota (svn), SCALE (hg), MOOSE (git submodules)

12Managed by UT-Battelle for the U.S. Department of Energy TriBITS Automated Package Dependency Handling

13Managed by UT-Battelle for the U.S. Department of Energy TriBITS Package Dependency Structure (e.g. Old Trilinos) RTOpTeuchosEpetraTriutilsThyraEpetraExt Required Dependence Optional Dependence

14Managed by UT-Battelle for the U.S. Department of Energy TriBITS Package Dependencies.cmake Files TRIBITS_PACKAGE_DEFINE_DEP ENDENCIES( LIB_REQUIRED_TPLS BLAS LAPACK LIB_OPTIONAL_TPLS Boost ) Teuchos TRIBITS_PACKAGE_DEFINE_DEP ENDENCIES( LIB_REQUIRED_TPLS BLAS LAPACK ) Epetra TRIBITS_PACKAGE_DEFINE_DEP ENDENCIES( LIB_REQUIRED_PACKAGES Teuchos ) RTOp TRIBITS_PACKAGE_DEFINE_DEP ENDENCIES( LIB_REQUIRED_PACKAGES Epetra ) Triutils TRIBITS_PACKAGE_DEFINE_DEP ENDENCIES( LIB_REQUIRED_PACKAGES Epetra Teuchos LIB_OPTIONAL_PACKAGES Triutils ) EpetraExt TRIBITS_PACKAGE_DEFINE_DEP ENDENCIES( LIB_REQUIRED_PACKAGES RTOp Teuchos LIB_OPTIONAL_PACKAGES EpetraExt Epera ) Thyra

15Managed by UT-Battelle for the U.S. Department of Energy TriBITS CI Testing: Change Epetra $./do-configure \ -D Trilinos_ENABLE_Epetra=ON \ -D Trilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES= ON \ -D Trilinos_ENABLE_TESTS=ON RTOpTeuchosEpetraTriutilsThyraEpetraExtLib OnlyLibs & Tests

16Managed by UT-Battelle for the U.S. Department of Energy TriBITS CI Testing: Change RTOp $./do-configure \ -D Trilinos_ENABLE_RTOp=ON \ -D Trilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES= ON \ -D Trilinos_ENABLE_TESTS=ON RTOpTeuchosEpetraTriutilsThyraEpetraExtLib OnlyLibs & Tests

17Managed by UT-Battelle for the U.S. Department of Energy TriBITS Usage of TriBITS Packages and Subpackages

18Managed by UT-Battelle for the U.S. Department of Energy TriBITS Software Engineering Theory about Packaging Package Cohesion OO Principles: REP (Release-Reuse Equivalency Principle): The granule of reuse is the granule of release. CCP (Common Closure Principle): The classes in a package should be closed together against the same kinds of changes. A change that affects a closed package affects all the classes in that package and no other packages. CRP (Common Reuse Principle): The classes in a package are used together. If you reuse one of the classes in a package, you reuse them all. Package Coupling OO Principles: ADP (Acyclic Dependencies Principle): Allow no cycles in the package dependency graph. SDP (Stable Dependencies Principle): Depend in the direction of stability. SAP (Stable Abstractions Principle): A package should be as abstract as it is stable. Problem: The Trilinos definition of a “Package” is not consistent with SE packaging principles most importantly the CRP Source: Martin, Robert C. Agile Software Development (Principles, Patterns, and Practices). Prentice Hall, 2003

19Managed by UT-Battelle for the U.S. Department of Energy TriBITS TriBITS Packages and Subpackages: Overview A A1A2 B C C1C2 TriBITS Parent Package: Collection of related subpackages Community of tightly integrated developers Unit of documentation, package-by-package CTest driver (single address) Downstream (SE) packages should not list parent package as a dependency! TriBITS Subpackage: Lightweight encapsulated collection of tightly related libs and tests/examples Lightweight use the options of the parent package Lists dependencies on upstream SE Packages & TPLs Primary unit for dependency management!

20Managed by UT-Battelle for the U.S. Department of Energy TriBITS TriBITS Packages and Subpackages: Dependencies A1A2 B C1C2 This is how the TriBITS dependency management looks at packages and subpackages => Packages and Subpackages are just are all just SE packages! The parent package is just an SE Package that depends (optional or required) on all of its subpackages New TriBITS packages should only have optional subpackages, have no required subpackages! Support for required subpackages is to maintain backward compatibility when packages are broken into subpackages (when optional packages are disabled). AC

21Managed by UT-Battelle for the U.S. Department of Energy TriBITS Depending only on subpackages: Example RTOp TRIBITS_PACKAGE_DEFINE_DEPENDENCIES( LIB_REQUIRED_PACKAGES TeuchosCore TeuchosComm TeuchosNumerics REGRESSION_ _LIST ) rtop/cmake/Dependencies.cmake $./do-configure -DTrilinos_ENABLE_RTOp=OFF –DTrilinos_ENABLE_Kokkos=OFF […] -- Setting Trilinos_ENABLE_TeuchosCore=ON because RTOp has a required dependence on TeuchosCore -- Setting Trilinos_ENABLE_TeuchosComm=ON because RTOp has a required dependence on TeuchosComm -- Setting Trilinos_ENABLE_TeuchosNumerics=ON because RTOp has a required dependence on TeuchosNumerics -- Setting Trilinos_ENABLE_TeuchosParameterList=ON because TeuchosComm has a required dependence on TeuchosParameterList […] Enabling all parent packages that have at least one subpackage enabled Setting Trilinos_ENABLE_Teuchos=ON because Trilinos_ENABLE_TeuchosCore=ON […] Final set of enabled packages: Teuchos RTOp 2 Final set of enabled SE packages: TeuchosCore TeuchosParameterList TeuchosComm TeuchosNumerics \ Teuchos RTOp 6 […] Final set of non-enabled packages: […] Kokkos […] 64 Final set of non-enabled SE packages: Gtest ThreadPool KokkosCore KokkosContainers \ KokkosAlgorithms KokkosExample Kokkos TeuchosRemainder TeuchosKokkosCompat \ TeuchosKokkosComm […] 142 Configure RTOp: NOTE: “Final set of enabled packages” means that at least one subpackage in that parent package is enabled and not that the parent package is explicitly enabled!

22Managed by UT-Battelle for the U.S. Department of Energy TriBITS Depending only on subpackages: Example Thyra TRIBITS_PACKAGE_DEFINE_DEPENDENCIES( SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS Core core PT REQUIRED EpetraAdapters adapters/epetra PT OPTIONAL EpetraExtAdapters adapters/epetraext PT OPTIONAL TpetraAdapters adapters/tpetra PT OPTIONAL ) thyra/cmake/Dependencies.cmake TRIBITS_PACKAGE_DEFINE_DEPENDENCIES( LIB_REQUIRED_PACKAGES TeuchosCore TeuchosParameterList TeuchosComm TeuchosNumerics RTOp ) thyra/core/cmake/Dependencies.cmake TRIBITS_PACKAGE_DEFINE_DEPENDENCIES( LIB_REQUIRED_PACKAGES ThyraCore Epetra TEST_REQUIRED_PACKAGES Triutils ) thyra/adapters/epetra/cmake/Dependencies.cmake TRIBITS_PACKAGE_DEFINE_DEPENDENCIES( LIB_REQUIRED_PACKAGES ThyraCore ThyraEpetraAdapters Epetra EpetraExt ) thyra/adapters/epetraext/cmake/Dependencies.cmake TRIBITS_PACKAGE_DEFINE_DEPENDENCIES( LIB_REQUIRED_PACKAGES ThyraCore Tpetra LIB_OPTIONAL_PACKAGES ThyraEpetraAdapters ) thyra/adapters/tpetra/cmake/Dependencies.cmake

23Managed by UT-Battelle for the U.S. Department of Energy TriBITS Packages that are broken into subpackages should have no libs of their own!  Original design of subpackages was not to allow a parent package to have any libs or test/examples of its own! (but was allowed by accident)  But packages can have their own tests/examples (if guarded correctly) Tight integration between the parent package and its subpackges: _ENABLE_ =ON => Equivalent to explicit enabling all subpackages _ENABLE_ =OFF => Equivalent to explict disabling all subpackages Tests/examples will be enabled for all enabled subpackges when the package’s tests/examples are enabled Parent package’s CMakeList.txt file is always processed when any subpackages are processed  Parent package’s options all defined (e.g. debug-mode checking, etc.)  Subpackages are always processed in the context of the parent!  Does not allow interleaving subpackages between parent packages! Question: Should we really support parent packages to have their own libs?  If so, some work is needed to specify this but it will be constrained in packages enable/disable logic … TriBITS Package and Subpackages : Details

24Managed by UT-Battelle for the U.S. Department of Energy TriBITS No more need to get individual copyright on Trilinos packages  Allows more freedom to break out new packages to get around the interleaving of subpackages! Consequences of move to GitHub on Dependencies

25Managed by UT-Battelle for the U.S. Department of Energy TriBITS Building Individual TriBITS packages as their own TriBITS CMake project

26Managed by UT-Battelle for the U.S. Department of Energy TriBITS Allow an upstream TriBITS Package to Build Independently Put ‘IF()’ statement in package’s top CMakeLists.txt for building as a TriBIITS project or a TriBITS package. Add /ProjectName.cmake Add /PackagesList.cmake Package dir: “. PT” Allow optional upstream packages to be missing TRIBITS_ALLOW_MISSING_EXTERNAL_PACKAGES() Add /TPLsList.cmake Point to standard TPLs under “${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/“ Or, copy critical FindTPL.cmake to /tpls/ and point to them there (no duplication) Or, simply give it a dummy directory and make it EX so that it will not get enabled. Everything else is unchanged!

27Managed by UT-Battelle for the U.S. Department of Energy TriBITS teuchos/cmake/Dependencies.cmake TRIBITS_PACKAGE_DEFINE_DEPENDENCIES( SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS Core core PS REQUIRED ParameterList parameterlist PS REQUIRED Comm comm PS REQUIRED Numerics numerics PS REQUIRED Remainder remainder PS REQUIRED KokkosCompat kokkoscompat PS OPTIONAL KokkosComm kokkoscomm PS OPTIONAL ) Dependencies file remains completely unchanged!

28Managed by UT-Battelle for the U.S. Department of Energy TriBITS teuchos/CMakeLists.txt IF (TRIBITS_PROCESSING_PACKAGE) # Processing TriBITS package! […] TRIBITS_PACKAGE_DECL( Teuchos ENABLE_SHADOWING_WARNINGS CLEANED ) […] TRIBITS_PROCESS_SUBPACKAGES() TRIBITS_PACKAGE_DEF() […] TRIBITS_PACKAGE_POSTPROCESS() ELSE(TRIBITS_PROCESSING_PACKAGE) # Processing as a TriBITS project CMAKE_MINIMUM_REQUIRED(VERSION FATAL_ERROR) INCLUDE("${CMAKE_CURRENT_SOURCE_DIR}/ProjectName.cmake") PROJECT(${PROJECT_NAME} NONE) SET(${PROJECT_NAME}_TRIBITS_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/tribits" CACHE PATH "By default assume Teuchos is in Trilinos") INCLUDE("${${PROJECT_NAME}_TRIBITS_DIR}/TriBITS.cmake") SET(TEUCHOS_STANDALONE_PACKAGE TRUE) TRIBITS_PROJECT_ENABLE_ALL() ENDIF(TRIBITS_PROCESSING_PACKAGE)

29Managed by UT-Battelle for the U.S. Department of Energy TriBITS teuchos/PackagesList.cmake TRIBITS_REPOSITORY_DEFINE_PACKAGES( Kokkos kokkos PT Teuchos. PT ) TRIBITS_ALLOW_MISSING_EXTERNAL_PACKAGES(Kokkos) Can build Teuchos by itself or with Kokkos => Need to fix gtest issue with Kokkos first (see Kokkos Issue #117)Kokkos Issue #117

30Managed by UT-Battelle for the U.S. Department of Energy TriBITS teuchos/TPLsLists.cmake, Trilinos/TPLsList.cmake teuchos/TPLsLists.cmake: TRIBITS_REPOSITORY_DEFINE_TPLS( BinUtils "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/" ST ARPREC "${${PROJECT_NAME}_SOURCE_DIR}/cmake/tpls/" ST QD "${${PROJECT_NAME}_SOURCE_DIR}/cmake/tpls/" ST MPI "${${PROJECT_NAME}_TRIBITS_DIR}/core/std_tpls/" PT BLAS "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/" PT LAPACK "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/" PT Boost "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/" ST QT "${${PROJECT_NAME}_SOURCE_DIR}/cmake/tpls/" ST Eigen "${${PROJECT_NAME}_SOURCE_DIR}/cmake/tpls/" ST ) Trilinos/TPLsLists.cmake: TRIBITS_REPOSITORY_DEFINE_TPLS( … ARPREC "packages/teuchos/cmake/tpls/" ST QD "packages/teuchos/cmake/tpls/" ST QT "packages/teuchos/cmake/tpls/" ST Eigen "packages/teuchos/cmake/tpls/" ST … ) FindTPL.cmake modules that are specific to the package should go in the package source directory!

31Managed by UT-Battelle for the U.S. Department of Energy TriBITS Configure of stand-alone Teuchos $ cmake \ -D TPL_ENABLE_MPI=ON -D CMAKE_BUILD_TYPE=DEBUG \ -DTeuchos_ENABLE_TESTS=ON -D BUILD_SHARED_LIBS:BOOL=ON \ $TEUCHOS_DIR … Processing Project, Repository, and Package dependency files and building internal dependencies graph... NOTE: Setting Teuchos_ENABLE_TeuchosKokkosCompat=OFF because package TeuchosKokkosCompat has a required dependency on missing package KokkosCore! NOTE: Setting Teuchos_ENABLE_TeuchosKokkosComm=OFF because package TeuchosKokkosComm has a required dependency on missing package KokkosCore! … $ time cmake. real0m4.567s user0m4.263s sys0m0.278s Get TriBITS from a) Trilinos (default), b) Install of TriBITS/tribits/ somewhere, c) Snapshot TriBITS/tribits/ into teuchos/cmake/

32Managed by UT-Battelle for the U.S. Department of Energy TriBITS TriBITS Snapshotting into Trilinos

33Managed by UT-Battelle for the U.S. Department of Energy TriBITS TriBITS Snapshots into Trilinos and local changes $ cd Trilinos.base/Trilinos/cmake/tribits/ $ git checkout --track origin/tribits_github_snapshot $../../TriBITS/tribits/snapshot_tribits.py $ git checkout master $ git merge --no-ff tribits_github_snapshot Changes made to Trilinos/cmake/tribits/ and committed to ‘master’ branch are not overwritten! Quick urgent changes can be made directly to Trilinos copy of TriBITS by Trilinos developers! TriBITS Maintainer will address merge conflicts with new snapshots Commits can be pulled off with ‘git format-patch’ and applied to the TriBITS repo using ‘git am’. Contributions to TriBITS must follow Contributing to TriBITS guidelines:Contributing to TriBITS All new features, bug fixes, and changes in behavior must have automated tests. All documentation is completed See: Trilinos/TriBITS development and sync:

34Managed by UT-Battelle for the U.S. Department of Energy TriBITS Summary

35Managed by UT-Battelle for the U.S. Department of Energy TriBITS TriBITS Partitioning and Dependencies TriBITS Core (tribits/core/): Core TriBITS package-based architecture for CMake projects includes configure, build, test, install, deploy (tarballs) for multi-repo projects. TriBITS Python Utils (tribits/python_utils/): Some basic Python utilities that are not specific to TriBITS (e.g. gitdist, snapshot_dir.py). TriBITS CI Support (tribits/ci_support/): Support code for pre-push continuous integration testing (e.g. checkin-test.py). TriBITS CTest Driver (tribits/ctest_driver/): Support for package-by-package testing driven by CTest submitting to CDash (e.g. TribitsCTestDriverCore.cmake). TriBITS Common TPLs (tribits/common_tpls/): Used by many independent TriBITS projects (e.g. FindTPLBLAS.cmake, FindTPLLAPACK.cmake, FindTPLHDF5.cmake, …) … TriBITS CTest DriverCMake python 2.4+gitCorePythonUtilsCI Support Common TPLs

36Managed by UT-Battelle for the U.S. Department of Energy TriBITS TriBITS Summary & Future work Summary: TriBITS enables interoperability and compatibility for large distributed projects TriBITS subpackages are designed for dependency management  Don’t depend on parent packages, depend on their subpackages!  Don’t add libs to a parent package with subpackages!  Update your Dependencies.cmake files! Upstream TriBITS packages can be made to build as own TriBITS CMake projects E.g. Teuchos + Kokkos TriBITS Core is only 1.4M and 10K lines of CMake code (no other dependencies) Future Work: Combining concepts of packages and TPLs for large meta-projects (TriBITS #63)TriBITS #63 High-level and tutorial documentation Several other issues (see TriBITS Issues and TriBITS Backlog)TriBITS IssuesTriBITS Backlog But once these are done => TriBITS will be a good candidate for a universal meta- build and installation system for a very large amount of CSE software!