API, Interoperability, etc

 Geoffrey Fox  Kathy Benninger  Zongming Fei  Cas De’Angelo  Orran Krieger*

 The (rapidly) changing technology suggest changing and standardizing APIs  Need to choose APIs to fit the emerging usage paradigm and fit with future technologies  API specifications, standards and implementations are needed for access and control of: ◦ compute (OCCI) ◦ Storage (CDMI-SNIA) ◦ Networking (implementations: OESS, FOAM, OSCERS)  There may need to be different APIs for users, (web/gateway/app) developers and sysadmins

 The challenge is exposing capabilities to the user  Commercial CI use is focused on object technology, which is different from academic implementations.  Academic community moving toward OpenStack (observation)  Commercial interfaces are often proprietary eg. AWS has building blocks, doesn't adapt to user needs  XSEDE Science Gateway is example for simplified user access as Gateway development benefits from APIs  Using APIs needs to be simple for the user!!

 Observation was made that there is a close relationship between specifying a system and specifying the workflow

 What are the critical aspects that are a clear win/how much of federation and distribution should be exposed to developers ◦ For performance ◦ To keep the exposed environment simple for usability  Exposing federation and distribution is powerful but complex  Note that industry does not expose the federation extraction to users

 Need research to understand when to give an API for a significant capability - when to hide the complexity and when to expose it  Need a list of use cases and scenarios to explore and investigate this

 Instrumentation and Monitoring (I&M)…  I&M is needed as part of the infrastructure and access should be available via API (of course!)  Data/logs from I&M capability need to be housed in a central repository (example is the XDMOD from XSEDE)  Note: some NSF funded projects don’t provide this for one reason or another; should be explicit in funding awards/agreements that this information should be provided in public domain (with sanitization as required)

 I&M continued  Standards and metadata for storage of the I&M data is needed  Need to investigate ways to correlate and understand this data  Discover and document full workflows for example from Jobs, data xfer, file system usage, application/tool usage, etc.  Need to investigate ways to visualize this complex data into information

 Cloud technology came first from industry  Research and education (R&E) community playing catch up and discovering new ways to provide and use cloud  Need to investigate and understand the R&E cloud business model from both the academic and NSF perspective  Would/does NSF allow awards to include cost of use of cloud resources like buying cluster equipment today?

 Orran gave his example of the cloud project funded by Commonwealth of Mass to put together a cloud resource supported by approximately 10 universities and industry  Also was mentioned that some universities are now investigating and interested in getting their compute from the cloud instead of local investments  this is in infancy and economic models need to be understood and discussed further