1. Code Generation
Tips
Tricks
Pitfalls
5/4/2019
A.Symons CiTR Pty Ltd

2. Why? Speed up development Remove bugs Consistency Less flexibility
DeltaX 8X
Remove bugs
Consistency
Less flexibility
DeltaX initially had 6 people doing 6 modules in 16 weeks. In the current DeltaX 2 people in 1 week developed a module.
Any bugs found can be fixed in the generator and thus be removed from all of the generated code.
All file names and function names can be made consistent. Also, variable names become consistent and easier to find - e.g., finding ifIndex in the MIB, source code and database. GUIs look the same.
All attributes can be supported, and things like help screens produced easily.
Less flexibility - Ties in with above - coders may want to capitialize differently, use ‘_’, drop characters. Also coders may not want to support all possible attributes / states, only those in the current test/design.
5/4/2019
A.Symons CiTR Pty Ltd

3. Why Not? May be difficult May not be appropriate
Readability of generated code
Debugability of generated code
Less flexibility
Really do want to have an initial prototype first. May have very difficult usage which is not consistent across input files.
Readability - code can often be mangled when generated (E.g., ROSE)
affects debugability.
Flexibility - Can be hard to put in special case code.
5/4/2019
A.Symons CiTR Pty Ltd

4. Example - DeltaX What High level design Detailed design API Document
SNMP MIBs &Database Access
SNMP Agent functionality
C++ SNMP & Corba Classes ...
GUI
Delta X is a Network management system - consists of
Design Documents
Api Documents
ILOG GUI
CORBA Config server
Postgres SQL database
SNMP Agent
Hardware API
5/4/2019
A.Symons CiTR Pty Ltd

5. Example - DeltaX EMS EMS SNMP Agent API Document High Level Design
MIB Master C Code
Subagent Set CB
Cache access
API calls
5/4/2019
A.Symons CiTR Pty Ltd

6. Example - DeltaX NMS NMS - C++ Corba + Ilog GUI IDL Corba Impl
DB schema & access
SNMP access
Ilog GUI
GUI C++
5/4/2019
A.Symons CiTR Pty Ltd

7. Example GUI
5/4/2019
A.Symons CiTR Pty Ltd

8. Lines 5/4/2019 A.Symons CiTR Pty Ltd
NMS Total is the total NMS. Not much was generated initially.
NMS New is the current phase of the NMS - really have been applying code generation
5/4/2019
A.Symons CiTR Pty Ltd

9. Percent 5/4/2019 A.Symons CiTR Pty Ltd
Significant parts of the API document, HLD, EMS and current NMS are now generated
Most of the current NMS is now able to be generated.
90-10% rule for NMS
5/4/2019
A.Symons CiTR Pty Ltd

10. Tips Metadata Utility printing executables Parsing executables
Integrated executables
Shell ‘here documents’
5/4/2019
A.Symons CiTR Pty Ltd

11. Metadata Describes the data
Want it ‘compiled in’ I.e., accessible in scripts
Source code
Database
Metadata parser & input file
Metadata is the heart of code generation.
This allows you to be able to print out the textual names of the attributes, structures and their types.
Do need to have a Metadata parser/compiler. This is not that difficult to implement in lex/yacc.
If you are specifying the input file format (e.g., internal API documentation) then keep one eye on being able to parse the file format to extract the metadata.
5/4/2019
A.Symons CiTR Pty Ltd

12. Parsing MibMaster etc. Bulk Code Metadata No Changes Generic
Difficult to debug/change
MM parses SNMP MIB definitions in ASNv1 and generates huge amounts of code (some files > 60 M)
This bulk code is so large that it becomes impractical to modify/debug, although it is possible.
This is mainly intended as the basis for other applications.
The parsers typically have printf type statements and themselves become difficult to follow.
These do however produce the metadata used in our other applications.
5/4/2019
A.Symons CiTR Pty Ltd

13. Integrated Based on intermediate metadata
Can do complex tasks - e.g., if then else
Difficult to modify
An example is the ilog generator
Put complex tasks in here, and put the rest in scripts
5/4/2019
A.Symons CiTR Pty Ltd

14. Utilities PrintMibs Prints metadata Print 17 items per attribute
Flexible - uses printf ‘%$’
Absolute key
5/4/2019
A.Symons CiTR Pty Ltd

15. Shell ‘here document’ Use existing example
Replace specifics with $variables
Enclose with cat << EOF … EOF
Easy to read, write and debug
New age of code generation
Shell scripts can be written with all of the logic at the top and then the ‘here’ document is essentially the source code with the specific names replaced with shell variables.
This produces a very readable code generator, which is also then easy to modify.
Similarly the generated code is very readable and debugable.
This is also quick to implement and also quick to modify/re-run.
Parts of the script can also be extracted for easier debugging.
5/4/2019
A.Symons CiTR Pty Ltd

16. Metadata Applications
E.g., GOA
Array of attributes
Switch statements/virtual functions
Very good for ‘raw’ applications
Low code base size
Easy to add new ones dynamically
Low code ‘*ability’
Difficult to add special cases
Templates and deep OO analysis also fits in here. CM used a lot of configuration to support new snmp agents.
The example of the GUI screen shows that it is not so easy to generate, either through scripts or through metadata a perfect implementation. I.e., extra information and logic is required for the perfect implementation, such as this table is an ATM link and many are required for an ATM PVC.
Low readability, writeability, debugability.
It is much easier to code with an explicit data structure, than with an array of attributes.
Probably need a rule based system to add in special cases.
5/4/2019
A.Symons CiTR Pty Ltd

17. Tricks How to identify Don’t try to be optimal initially
Doing the same thing over and over
Use generation when possible.
Only use generation when possible.
Cut and Paste for small amounts.
Don’t try to be optimal initially
Need a lot of something - a lot of screens, database tables, documents …
Use metadata based solutions/templates when you are doing exactly the same thing for each one - I.e., no modifications, no special cases.
Use generation when doing similar things but some special cases.
Use cut and paste when some can be generated but most is special case code.
90-10 % rule - generate the 90% case code - easy. Do the 10% difficult code by hand.
5/4/2019
A.Symons CiTR Pty Ltd

18. Pitfalls Modifying Code Code understandability Stub Files RCS
#ifdef wrappers
Derived classes
Registered Callbacks
Don’t generate as part of the normal build
Code understandability
Generated code is good if you are not modifying it. However, problems occur as soon as you do modify the code. Especially if multiple coders.
Stub files for code you know you are going to have to hand code.
You really should keep all of the changes in one file, preferably not a generated file.
Patch shell scripts
RCS all code that you do modify - so that you have a chance of getting it back.
If all of the code is generated then you may not understand what is going on. Especially if you have to fix the code! Really do need good documentation.
5/4/2019
A.Symons CiTR Pty Ltd

19. Conclusions Generation can reduce time
Use 90% generated 10% hand coded
Use metadata based for ‘raw’ apps
Shell scripting is fast and readable.
Take care when modifying generated code.
5/4/2019
A.Symons CiTR Pty Ltd

20. References MM hptools/yacc PrintMibs mibmaster/008.Tools/print_mibs
GOA mibmaster/007/src/monitor/goa
gen nms nc867.15/007/src/gen_nms
ems nc867.15/007/bin
5/4/2019
A.Symons CiTR Pty Ltd