1. Database Developers Forum – APEX
APEX Experience from the Accelerators Controls and Operations Data Management in the Beams Department
Database Developers Forum – APEX
Antonio Romero Marin, Aurelien Fernandes, Jose Rolland Lopez De Coca, Nikolay Tsvetkov, Zereyakob Makonnen, Zory Zaharieva
BE-CO

2. Contents
Introduction to the Controls environment and some of the services using APEX
Interesting features for developers
APEX_COMMON Framework
Plugins & custom components, look and feel features, etc.
Best Practices
Conclusion 2
Z.Zaharieva Controls Configuration Service - LS1 Plans EDMS doc:

3. Introduction to the Accelerators Data Management
Complex and challenging area
Federation of various databases, e.g.
Accelerators Controls Configuration
Diagnostics and Monitoring (DIAMON)
Alarms (LASER)
Accelerators Entities and Signals Naming
Logging and Measurements, etc.
Strategy
Relational DB technology – Oracle DBs for all technical data
Data-driven applications and APIs - Oracle technology stack
Reliable database services and user interfaces (24/7/365)
On-line usage of database services for the accelerators Control and Operation

4. APEX Development throughout the years
Starting with HTMLDB
beginning of 2005, HTMLDB v.1.6
Need for rapid application development (Agile programming)
fast prototyping and short time to production deployment
quickly respond to new user requirements
Ready to use templates, widgets and page components
professional look-and-feel with minimum effort on design – concentrate on the application itself
The nature of the interfaces to be implemented
web-interfaces, database-centric
APEX supports client-side scripting for more dynamic / interactive / feature-rich interfaces

5. First Trials of APEX
Data browsing interfaces and portal for the Accelerators Entities and Signals Naming Database
The Naming DB – dictionary for the accelerators complexes
Equipment types
Equipment positions in the accelerators
Settings and observables (signals)
Read-only interfaces to the data
LHC Equipment catalogue
SPS equipment catalogue
PS equipment and layout components catalogue
Signals Simple Extraction Interface
Signals Experts Extraction Interface
Naming Domain Portal

6. Accelerators Naming DB Interfaces
Data-browsing Interfaces (window-on-data) – development from scratch
low level of complexity
no transactions
some business logic to get the data out
user authentication and simple user authorization
easy extraction of data into XML/CSV files

7. Portal Pages and News feeds
Simple portal pages
Access to other related interfaces, documentation, help pages
Dynamically constructed using data stored in tables
News, RSS feeds, News Archive, etc.

8. History Log Browsers History logging in our dbs
All data modifications are recorded (to know who did what and when)
History Log Browser objectives
Access the history log for any table in the database for which history is enabled
Provide simple search capabilities for the history logs
Provide a summary for the changes of a given record based on the PK
Completely dynamic configuration
List of tables, list of columns per table, primary keys for tables (USER_ tables)
Dynamic PL/SQL embedded in APEX regions
Portable solution
Deployed to several of the databases, which are using the same mechanism for keeping history without any changes to the interface components

9. History Log Browsers

10. Controls Configuration Service
The heart of the Controls System – the basis for the Configuration Management of all Controls System components in the entire accelerator complex
Provides data for all configuration items and their relationships, required for the correct functioning of the Controls System
All controls devices (>105,000) and parameters (> 2,000,000)
Hardware and software configuration of all Front-End Computers (>3,500), Drivers generation
The Accelerators Timing System, Operators Consoles, etc.
Service with 30 years of history
Supporting the requirements of the entire accelerator complex – LHC, SPS and PS complex

11. Controls Configuration Data Browser
An existing PL/SQL OWA application with ~ 100 reports
existing PL/SQL code base in the CCDB
a complete re-engineering of the application was undertaken in 2006
possibility to reuse some of the PL/SQL code
‘thick’ database layer – PL/SQL packages in the database
The first big scale data browsing application to use APEX
Nowadays >220 reports used by a user community of >300 people - thanks to
the ease of APEX reports development
Introduction of APEX user interactive reports
set of OWA PL/SQL reports with PL/SQL code base in the database

12. Alarms (LASER) Database
LASER (Alarms) - capturing, storing and notification of anomalies for the whole accelerator chain and technical infrastructure
Alarms DB - 3 different database areas
Pre-defined alarm definitions
User configurations for the alarms displays
Time-stamped run-time alarms events
First APEX data editing interfaces
fast development
evolution of APEX
suite of Data Management Tools (7 Editors)
developed from scratch
give users the possibility to explore their data and maintain it

13. Controls Configuration Data Editors
Data-manipulation interfaces
complexity of the applications due to transaction management
complex business logic to handle modifications of the data
stringent data access rights – e.g. specific authorization modules, virtual private db, etc.
Existing web-deployed, rich clients, using Oracle ADF
Renovation of the existing interfaces and providing new ones with APEX since beginning of 2011

14. Controls Configuration Data Editors
The first relatively big scale and complex editing applications to be developed / migrated to APEX
So far - 15 Data Editors (> 100 forms), ~ 250 user community
4 editors remaining to be renovated

15. Internal developers applications
Due to the ease of development, the developers are using APEX to create applications for managing some of the developers specific processes
Admin Editor for Service Interventions
DB Schema Change Management Editor
APEX _COMMON Metadata Editor

16. APEX in the Controls Environment nowadays
Specific reporting tools
used for CCS, Laser, Diamon, Accelerators Naming, Measurements
> 300 reports
Generic reporting modules and applications for managing the developers tasks
History Browser; Schema Change Management; Admin Interventions Config, etc.
Portals and news pages with RSS feeds
Data editors
used for CCS, Laser, Diamon, Layout, R2E, SM18 Data Analysis
> 25 editors (> 200 editing forms)

17. Using APEX for the accelerators data management
Flexible environment for development
Allows you to do a lot of things
It is up to the developers to provide a structured approach to application design
Complete integration with our databases (Oracle, PL/SQL)
Provides a lot of features and components ‘out of the box’
Fast and easy development
We can do even better
Custom libraries and components
Best practices, guidelines and coding standards
Speed and Scalability

18. APEX_COMMON framework
APEX built-in tabular forms
APEX manual tabular forms
APEX_COMMON framework
Examples and demo
Evolution of the framework 18
Z.Zaharieva Controls Configuration Service - LS1 Plans EDMS doc:

19. Tabular forms – what is it?
Grid to edit multiple rows or records at once

20. Built-in tabular forms in APEX
Very easy creation with APEX wizard
Provide built-in functionality for insert/update/delete
Other built-in features (validations, partial refresh, CSV export…)

21. Limitations of the APEX built-in tabular forms
Really easy to create, but there are important limitations
Just ONE tabular form per page
No Parent-Child tabular forms
Conditional rendering is not possible
Does not keep changes across page submits

22. APEX manual tabular forms
APEX report using the APEX_ITEM package
Usually using also APEX collections
temporary storage of data within a user session
Require more work than normal tabular forms but overcomes the built-in tabular form limitations
It also adds new features and possibilities
e.g. use of interactive reports

23. Manual tabular forms – workflow processing
Steps to process a manual tabular form - using collections
Create and load data into collection from the table
Create query using APEX_ITEM package
Update the collection
Commit changes in the DB
Client-side
Report on Collection 2 3
Server-side
APEX Collection
Page Rendering
Page Processing 1 4
Tables

24. APEX manual tabular forms – creation process
You have to define many components for each collection
APEX global processes, JavaScript functions, PLSQL procedures, etc. …
Error prone
Collection columns names like c002, page item names like f01
If you want to use AJAX it adds more complexity
Difficult to maintain the page and to follow its evolution, specially when there are multiple developers

25. Our environment Large number of applications and pages
Need of rapid development
Many parent-child tabular forms in the same application page
Custom logic that APEX standard components do not cover
e.g. conditional item rendering
So we created APEX_COMMON to make those tasks easier

26. What is APEX_COMMON?
A framework to ease the development of manual tabular forms
Creation
Manipulation
Maintenance
Modify behaviour of multiple pages in a systematic and fast way
Reusability of components
Set of generic components (JavaScript libraries, PL\SQL packages, APEX processes, etc.)
Generates code for the manual tabular form
Use standard APEX libraries

27. APEX_COMMON Architecture Overview
APEX Application
Database User Schema
APEX_COMMON JavaScript API
APEX_COMMON
metadata
AJAX On Demand processes and
POST submits
APEX_COMMON PL/SQL API

28. Architecture and Components
DB side
Metadata in tables for the manual tabular form definitions
APEX_COMMON PL/SQL package
create collections
manipulate collections
commit changes in DB target tables
Global APEX Application processes
Interfacing with the db

29. Architecture and Components
Client side
Global APEX dynamic actions (functions)
Global JavaScript library
Generic functions to interact with the data (upload data from page to collections, duplicate rows in tabular form…).
Other generic features (highlight rows)
APEX Application
APEX_COMMON JavaScript API

30. APEX COMMON CCDB Tool Tool to support APEX_COMMON
Declare tabular forms definitions
Automatic code generation
-Create objects using APEX framework in desired page.
-Copy and paste code from APEX_COMMONS CCDB Tool.

31. Examples and DEMO
Delete ”setup in APEX page” part??

32. Controls Configuration Hardware Editor
Front-End Computer configuration form (Crates and Modules)
multiple child forms
Dynamic field types
(e.g. field editable or not depending on value of other column)

33. Controls Configuration Hardware Editor
Hardware Type Properties
custom logic

34. Controls Devices Editor
Virtual Classes form
3 level parent-child forms

35. Demo Create a manual tabular form 1. Without APEX_COMMONS
2. With APEX_COMMON

36. Evolution of the APEX_COMMON
During the last 9 months since the creation of the APEX COMMON
Optimizing performance when using collections and large data sets
save only the changes in the collections
Management of APEX collection
18 => 48 columns per manual tabular form
Shared code in Page 0
Highlighting, Check all, Duplicate, Add Row…

37. Features under development – part1
Editable Interactive Report
All common dynamic actions in Page 0 (e.g. load collection, update collection) => Less code in the pages

38. Features under development – part2
Summarize user actions before commit
More auto generation
=> Fill information from database dictionary
Capture page exit event and warning user about pending modification
Optimization of server calls => JSON manipulation in PL/SQL and JavaScript

39. APEX_COMMON Summary Multiple Tabular Form per page
APEX 5.0 will NOT have support for multiple tabular forms
APEX 5.1 will introduce a new region type that allows it
It is in the statement of direction but… will it really be there?
Low cost of code evolution and maintenance (code standardization)
Easy to introduce custom logic in the pages
It is a portable solution – you can use it
APEX_COMMON Wiki tutorial

40. Interesting Features for APEX Developers
Cascading LOVs in tabular form
Plugins
Advanced manual tabular form with dynamic number of columns
Bulk data loading features 40
Z.Zaharieva Controls Configuration Service - LS1 Plans EDMS doc:

41. Custom made components for APEX
What do we gain from them?
make the application more user friendly
prevent the users of making mistakes
Why is it interesting to present them today?
they are not built-in APEX components
could be useful for other APEX developers

42. Cascading LOVs in tabular form
What does it mean cascading List Of Values?
the selection of the second LOV depends on the first one

43. Cascading LOVs in tabular form
How does it look in our data editors?

44. Cascading LOVs in tabular form
How do we do it ?
JavaScript
jQuery
JSON
OnDemand PL/SQL process
Find the Wiki page on the following link:

45. Plugins – usage and observations about them
The most interesting and often used ones
Custom pop-up plugins for selection
Save before exit plugin
Advantages of the custom pop-up plugins
possibility of quick search in multiple columns
shows structured data based on a custom SQL query
possibility of multiple columns update
doesn’t open new browser window
Find “Custom Pop-up plugin for selection” Wiki page

46. Custom pop-up plugin for data selection
APEX provided one

47. Custom selection pop-up

48. Save before exit plugin
Transactional model in our data editors
auto-commit is not an option
the user decides when to commit or rollback the changes
Why do we need to check for changes?
prevent losing data modifications
ignore changes in the search region
Find “Save Before Exit” Wiki page
Discuss to combine with the next slide

49. Save before exit plugin

50. Save before exit

51. Save before exit

52. Advanced manual tabular form with dynamic number of columns
What was the challenge?
columns generated dynamically based on data stored in a table
How do we do it?
build SQL dynamically using pivot in order to transform data stored into table rows and map them to columns
use “Generic Column Names” tabular form option
save the data in the DB using unpivot

53. Dynamically generated columns in a tabular form
Table2 LogicalEvents – stored in rows and presented in columns.
Devices from the same class have the same number of LogicalEvents
Table1 Devices – names of devices stored in rows
Table3 Event Configuration – data stored in rows, based on the device and LogicalEvent

54. Bulk data loading features
What is bulk data loading?
Who benefits from providing this functionality?
users are able to import big sets of data using CSV files
support team reduces the number of requests
We improved the standard APEX component
to provide more structured information
to add different highlighting for all the actions
to show result details for all updated and inserted rows
Find the Wiki page on the following link

55. Bulk data loading features

56. Bulk data loading features

57. Bulk data loading features

58. Bulk data loading features

59. Interesting Features for APEX Developers
From static to dynamic contents
Look and feel features
Collaborative development
Functional testing
The good, the bad and the ugly
Best Practices 59
Z.Zaharieva Controls Configuration Service - LS1 Plans EDMS doc:

60. How to move from static to dynamic contents(1)
Explain how in Oracle Apex a page is processed.

61. How to move from static to dynamic contents(2)
Dynamic Action 
Region refresh, Set value, Show, Hide, Clear
Handling of evens
Click/doubleClick
Load/Unload page
KeyPress/MouseButtonPress
Change …
Report partial refresh (Page Partial Refresh - PPR)
Report partial page refresh aka PPR is the Apex feature that allows a user to refresh only the portion of page containing the report whenever a pagination or column sort link is clicked.
This effect is achieved by means of built-in AJAX code.

62. How to move from static to dynamic contents(3)
JQuery (APEX > 4.0)
Standard APEX API
Dynamic call to application process
// create ondemand request
var req = new
apex.ajax.ondemand(‘apex_application_process_name',function() {
if(p.readyState == 1||p.readyState == 2||p.readyState == 3) {
//process has not yet finish… }
else if(p.readyState == 4) {//process has finish….. }
else {return false;}
});
// send arrays (tabular form columns)
req.ajax.AddArrayItems(apex.jQuery('[name=f03]'),1);
// submit the ajax process
req._get();
Region refresh
$('#report_static_id').trigger('apexrefresh');
Set and get Item value
$s(‘P2_ITEM_ID’,4) , $v(‘P2_ITEM_ID’)
For the most advanced people or those that have knowledge in Jquery.

63. Look and feel features(1)
How to keep the Tabular form features and reduce the number of columns for very large tables/forms
show the data in a different way
The Tabular form allow user updating data in bulk this is the only APEX component that allow to do that.
It’s a pity and a waste of time not to use the functionality of this components just because the user would like to see the data in a different way or because the number of columns is to big to fit in one page.

64. Look and feel features(1)

65. Look and feel features(2)
Space optimization on a page
Space optimization.
Some time I’ve the feeling that Apex tries to use as much space as it can.
If you look at report on the left you will see that the first column is taking all the space without leaving much to the second one.
Configuring properly the CSS is possible to overcome this problem and gain some space.

66. Collaborative development useful features
Use of a base Application
Authentication Scheme
Authorization Scheme
Theme
Navigation bar
Plugins
Process for common management of objects
JS files, Images, workspace theme
Page Lock feature
Because our service is providing different applications to the Controls Configurations community we decided to use a base Application as a template to start any development of new application.
One person is in change to keep the base application in sync with the latest version of the different components used.
Page lock features :As our team provide support for the existing application can happen that an urgent bug fix could be overwritten by another developer that is working on the same page.

67. Functional testing for web interface
Investigated into testing framework Watir WebDriver
It uses Selenium WebDriver
Higher level API than Selenium WebDriver
Used by industry leading companies
Open source
Integration with Bamboo (standard BE-CO Continuous integration testing framework) ( )
For the testing of our web interface after some investigation on different product we choose Watir.
Badboy: but was standalone product and not possible to integrate with other system.
Jmeter: Can be used for performance tests. It does not take into account different web browsers behaviors + visualization issues.
We already have a set of class written in Rudy used for the prototyping of the first tests.
Watir web Drive is a wrapper of Selenium WebDrive that allow testing of web interfaces.
Selenium WebDrive : is a set of different software tools each with a different approach to supporting test automation. The entire suite of tools results in a rich set of testing functions specifically geared to the needs of testing of web applications of all types. These operations are highly flexible, allowing many options for locating UI elements and comparing expected test results against actual application behavior. One of Selenium’s key features is the support for executing one’s tests on multiple browser platforms.
One of the interesting things is that is simple to use because it offer a high level API . You can create modular test creating modules that can be called form different set of tests.
It offer the possibility to automate the testing of the interfaces integrating it with Bamboo using Ruby plugin.

68. The good, the bad and the ugly
Advisor 
Utilities-> Advisor
Advisor does not check the application process 
Passing parameter via URL 
(:) must be encoded before sending it
Two tabs browser problems 
no standard solution provided in APEX yet for this known web applications issue
not possible do define the scope of page item
Best practices: Use aliases for APPLICATION and PAGES it will easy the maintenance of the links used to navigate between pages, specially if you have interconnected application.
Reduce at the minimum the code: Move the logic into packages. This will easy the maintenance task because will not require the deployment of the all application unless the changes affect the interface.
I’ve add a link of a presentation form Steven Feuerstein which is very interesting where it gives his advices and recommendations about this topic.

69. Some Best Practices Use aliases at application / page level
Making the application portable
Use of APEX-provided substitution strings - &APP_ID., &APP_ALIAS., etc.
Custom-defined substitution strings – application definition level
Reduce at the minimum the code in the APEX pages
Move the PL/SQL code into DB packages.
Best Practices For PL/SQL Development in Oracle Application Express (Steven Feuerstein)
Best practices: Use aliases for APPLICATION and PAGES it will easy the maintenance of the links used to navigate between pages, specially if you have interconnected application.
Reduce at the minimum the code: Move the logic into packages. This will easy the maintenance task because will not require the deployment of the all application unless the changes affect the interface.
I’ve add a link of a presentation form Steven Feuerstein which is very interesting where it gives his advices and recommendations about this topic.

70. Contents
Introduction to the Controls Environment and services using APEX
APEX Commons Framework
Interesting features for developers
Plugins & custom components, look and feel features, etc.
Best Practices
Conclusion 70
Z.Zaharieva Controls Configuration Service - LS1 Plans EDMS doc:

71. Using APEX for the accelerators complex
APEX has evolved a lot during the last 9 years since we started using it in the domain of the Accelerators Data Management
Could be used from small to relatively large/complex applications
Has proven itself able to fulfill the stringent application needs of the accelerators data management
2014
2005

72. Using APEX for the accelerators complex
Quality assurance rules and guidelines in order to ensure scalability
APEX guidelines – workspace structure, version control, naming conventions, etc.
PL/SQL and SQL guidelines
coding standards and best practices
Providing common components – customized objects and additional facilities for even faster development
Uniformity within the applications
Faster development
Easy maintenance

73. Questions? Thank you for your attention!