LSST DC2 Post-Mortem February 5–6, 2008 1 DataProperty Generally works Uses: –FITS headers –Events –Persistence parameters (additionalData) 1.

Slides:



Advertisements
Similar presentations
Lectures on File Management
Advertisements

Two-Pass Algorithms Based on Sorting
MapReduce Online Created by: Rajesh Gadipuuri Modified by: Ying Lu.
A Crash Course Python. Python? Isn’t that a snake? Yes, but it is also a...
1 DPS for DC2 Summary Model Implementation –Pipeline & Slice in Python and C++ Stage Loop, Policy configuration, Event handling in Python MPI env and communications.
Data Types in Java Data is the information that a program has to work with. Data is of different types. The type of a piece of data tells Java what can.
1 Week 12 FAT32 Utility Operations Guide Classes COP4610 / CGS5765 Florida State University.
Concepts of Version Control A Technology-Independent View.
Microsoft Access 2003 Introduction To Microsoft Access 2003.
Hands-On Microsoft Windows Server 2003 Administration Chapter 5 Administering File Resources.
XP New Perspectives on Microsoft Office Access 2003, Second Edition- Tutorial 1 1 Microsoft Access 2003 Tutorial 1 – Introduction To Microsoft Access 2003.
CSE 250: Data Structures Week 3 January 28 – February 1, 2008.
Introducing JavaBeans Lesson 2A / Slide 1 of 30 JDBC and JavaBeans Pre-assessment Questions 1.Which of the given symbols is used as a placeholder for PreparedStatement.
CIT 590 Unit testing. Agenda Debugging attempt 2 (because I am stubborn) What is unit testing Why? Unit testing framework in Python.
.PDF = Portable Document Format  Providing a common interface for opening documents sourced in a wide range of applications  Collaborating with others.
1 Spidering the Web in Python CSC 161: The Art of Programming Prof. Henry Kautz 11/23/2009.
Chapter 10 Classes. Review of basic OOP concepts  Objects: comprised of associated DATA and ACTIONS (methods) which manipulate that data.  Instance:
JavaScript, Fourth Edition
Dale Roberts Department of Computer and Information Science, School of Science, IUPUI CSCI 230 Information Representation: Negative and Floating Point.
The Pipeline Processing Framework LSST Applications Meeting IPAC Feb. 19, 2008 Raymond Plante National Center for Supercomputing Applications.
Handling Lists F. Duveau 16/12/11 Chapter 9.2. Objectives of the session: Tools: Everything will be done with the Python interpreter in the Terminal Learning.
Copyright © 2012 Pearson Education, Inc. Publishing as Pearson Addison-Wesley C H A P T E R 2 Input, Processing, and Output.
Object Oriented Programming Elhanan Borenstein Lecture #4.
Databases Lesson 5.
Windows NTFS Introduction to Operating Systems: Module 15.
Arrays BCIS 3680 Enterprise Programming. Overview 2  Array terminology  Creating arrays  Declaring and instantiating an array  Assigning value to.
Collecting Things Together - Lists 1. We’ve seen that Python can store things in memory and retrieve, using names. Sometime we want to store a bunch of.
JAVA SECURITY BASIC NETWORKING MULTITHREATING Deniz HASTORUN
XP New Perspectives on Microsoft Access 2002 Tutorial 1 1 Microsoft Access 2002 Tutorial 1 – Introduction To Microsoft Access 2002.
Slides prepared by Rose Williams, Binghamton University Chapter 5 Defining Classes II.
Built-in Data Structures in Python An Introduction.
Software Documentation Section 5.5 ALBING’s Section JIA’s Appendix B JIA’s.
Database structure and space Management. Database Structure An ORACLE database has both a physical and logical structure. By separating physical and logical.
Module 3 Configuring File Access and Printers on Windows 7 Clients.
Module 3: Configuring File Access and Printers on Windows 7 Clients
Comp 248 Introduction to Programming Chapter 4 & 5 Defining Classes Part B Dr. Aiman Hanna Department of Computer Science & Software Engineering Concordia.
Persistent Object References in ROOT1 Persistent Object References in ROOT I/O Status & Proposal CMS-ROOT meeting CERN- November 27 Ren é Brun ftp://root.cern.ch/root/refs.ppt.
Overview The Basics – Python classes and objects Procedural vs OO Programming Entity modelling Operations / methods Program flow OOP Concepts and user-defined.
Python Primer 1: Types and Operators © 2013 Goodrich, Tamassia, Goldwasser1Python Primer.
Programming with Java © 2002 The McGraw-Hill Companies, Inc. All rights reserved. 1 McGraw-Hill/Irwin Chapter 5 Creating Classes.
0 Processing Dollar ($) Receiving entries using webBASIS.
Unit 7 Number Systems and Bases Presentation 1Binary and Base 10 Presentation 2Adding Binary Numbers Presentation 3Subtracting Binary Numbers Presentation.
5 Feb 2008 Roberta Allsman LSST Corporation LSST Data Management DC2 Post-mortem.
CS4432: Database Systems II Query Processing- Part 2.
Operators and Expressions. 2 String Concatenation  The plus operator (+) is also used for arithmetic addition  The function that the + operator performs.
The world leader in serving science Overview of Thermo 21 CFR Part 11 tools Overview of software used by multiple business units within the Spectroscopy.
LSST DC2 Post-Mortem February 5–6, DataProperty Generally works Uses: –FITS headers –Events –Persistence parameters (additionalData)‏ 1.
PHP vs. Python. Similarities are interpreted, high level languages with dynamic typing are Open Source are supported by large developer communities are.
Presentation OLOMOLA,Afolabi( ). Update Changes in CSV/SVN.
Outline of the Objects used in Analysis Base Classes (module) Compound Classes (module) Utilities/Helpers (module) MrsData (mrs_data) Isa: MrsPack Container.
Chapter 11: Sequential File Merging, Matching, and Updating Programming Logic and Design, Third Edition Comprehensive.
Lesson 6-Using Utilities to Accomplish Complex Tasks.
LECTURE 2 Python Basics. MODULES So, we just put together our first real Python program. Let’s say we store this program in a file called fib.py. We have.
INCREMENTAL AGGREGATION After you create a session that includes an Aggregator transformation, you can enable the session option, Incremental Aggregation.
PROGRAMMING USING PYTHON LANGUAGE ASSIGNMENT 1. INSTALLATION OF RASPBERRY NOOB First prepare the SD card provided in the kit by loading an Operating System.
Quiz 3 Topics Functions – using and writing. Lists: –operators used with lists. –keywords used with lists. –BIF’s used with lists. –list methods. Loops.
Python: File Directories What is a directory? A hierarchical file system that contains folders and files. Directory (root folder) Sub-directory (folder.
Today… Files from the Web! Dictionaries. Lists of lists. Winter 2016CISC101 - Prof. McLeod1.
Lists/Dictionaries. What we are covering Data structure basics Lists Dictionaries Json.
FILES AND EXCEPTIONS Topics Introduction to File Input and Output Using Loops to Process Files Processing Records Exceptions.
Tim Hall Oracle ACE Director
Python’s Modules Noah Black.
Containers and Lists CIS 40 – Introduction to Programming in Python
Call Stacks, Arguments and Returns
SQL – Python and Databases (Continued)
Programming Logic and Design Fourth Edition, Comprehensive
CISC101 Reminders Slides have changed from those posted last night…
Chapter 8 Collection Types.
Example of A* A (4,0) 4 OPEN: (A4) CLOSED: ().
Python Primer 1: Types and Operators
Presentation transcript:

LSST DC2 Post-Mortem February 5–6, DataProperty Generally works Uses: –FITS headers –Events –Persistence parameters (additionalData) 1

LSST DC2 Post-Mortem February 5–6, DataProperty boost::any problems Must know type before extracting value Need functions to get an integer/real/string from anything compatible Need function to get a dynamically-typed Python object Added explicitly typed construction/extraction functions for Python 2

LSST DC2 Post-Mortem February 5–6, Added and excess functionality Added function to combine two DataProperties –For updating WCS parameters findUnique() used everywhere –Ability to have multiple identical keys not useful SupportFactory overdesigned for DC2 –None of functionality used; class itself not used sometimes 3

LSST DC2 Post-Mortem February 5–6, Policy Very successful JSON deprecated; should be removed File-include functionality not used yet Several policy files per pipeline and per run –Policies used only determinable from interpreting other Policies –Therefore need to persist all for provenance –Master Policy not currently saved (will be in ticket #315) Has get()/getArray() functions for Python Does not try to support float, int64_t Should be merged with DataProperty –But marked read-only, since modifying Policies is undesirable 4

LSST DC2 Post-Mortem February 5–6, Utility Functions These functions are in various “utility” modules; they belong in DataProperty/Policy or a common location instead: –fw::getInt64FromAny() — Extract an int64_t value from a boost::any if it holds a signed integral type –dps.Utils.getDPValue() — Extract a value from a DataProperty into a Python dynamically-typed object –dps.Utils.dataPropertyToDict() — Convert a DataProperty into a Python dictionary –ap::extractRequired() — Extract a value from a DataProperty, throwing an exception if not found –fw::extractPolicyString(), ap::extractPolicyString() — Extract a value from a Policy if the Policy exists, otherwise return a default value 5