Exposing Gen

I attended a session yesterday in which I was discussing the appropriate way of exposing Gen functionality into an SOA. I discussed three ways of getting Gen functionality “out” of a Gen environment.
One of the questions that I fielded was “How easy is it to take Gen functionality and expose it at different levels”.
“Different levels” in this case means - either at the procedure step level or at the action block level.
In principle, it is the same, but it transpires that in our organisation, where we have complex procedure steps aligned to a single business process, consisting of MANY hundreds of action blocks, each containing a piece of common functionality - basic field validation, business rules etc etc… that the appropriate exposition level is BOTH, BUT IN DIFFERENT WAYS !!
For simple, quick pieces of individual validation which do NOT require any database access, we thought that we would generate raw java code and deploy that straight into our web application sever infrastructure (hand-crafted java), for speed, but for more complex pieces of validation and business logic, and the final database update stuff, we would expose a web service, via the Java proxy of the full procedure step, minus any appropriate validation that we had already deployed to the web application server. Bearing in mind that the back end deployment is deployed on a cluster of Unix boxes in C, again for speed.

Thinking about this, it seems that reuse is ALWAYS achievable, but we have to be careful WHAT we reuse and in which way, and that a number of deployment decisions have to be made as to HOW the deployment is done.

Sometimes, picking small bits of existing business transactions and deploying them to the web is appropriate, since the paradigm of web data entry is usually different to “traditional office keying”, where a web screen consists of a small number of entry fields, and validation “as you go”, and a final “submit” where the user shouldn’t be told “by the way you have a problem on screen 1 (of 10)” - they should be told as they go that there is an issue. Traditional office keying, especially in a high-throughput environment is a “key the lot, then check it” approach.

Picking bits of business logic and validation from an existing business transaction that is to be web-enabled can be done in a number of ways (the way described is just one) allows maximum reuse, maximum flexibility in deployment (not discussed yet !!) and uniformity of business rules and inplementation - demonstrating yet again that Gen can abstract an organisation from technology, but still be confident that multiple technologies can implement consistent rules, since the rules are stored in a model !!!!

Should Gen allow database-specific settings ?

Following on from the last post on database independance, it got me thinking…..

 Since there are DB2-specific settings available in Gen could we not have Oracle or Datacom settings (or any other database that Gen supports ?) 

What settings would we need ?

 Well…..

…as a for instance, very often IU have had to hack some code to allow an “explain plan” to be set for a specifi piece of SQL which we perceive to be non-performant - I’d like to rebuild a piece of batch or online code with the ability to switch on or off this feature for a specific READ statement when I build - the syntax may be different for each database, but the generation options I select lead to database-specific pieces of SQL being generated anyway depending on the target database - so why not this ?

 … as another for instance,  when I am setting up a new entity in Gen and need to retransform to get the DDL, I always have to have my DBA’s modify the DDL to add in their clauses for creating the corresponding table in a certain tablespace or a specific disc for optimum performance - similarly with indexes, we have to amend the DDL to allow for specific fragmentation strategies.

It would, I admit, lead to an issue when changing database technology, but then, the planning for disc layouts and index structures HAS to be redone anyway, so why not capture that information in a Gen model and have the whole system self-contained.
This could be seen as a counter-argument to Gen’s flexibility in that system-specific settings and physically-dependant settings are in the Gen model, but at least all of them are then in the same place. Any time a relayout of the database on the physical discs is carried out for reasons of performance or new hardware procurement, then the Gen model is changed and retransformed and the new DDL can be produced.
My point is that ALL aspects of the system can then be controlled from within the model.

It would be interesting to hear feedback on this, as it is a continuing discussion within my own organisation as to where this “database stuff” is best kept…….

Database Independance

Is it really true that Gen can deploy into different environments ?

Yes - it is, but what happens when you change the database technoloigy that is underlying the application ? Does Gen help here ?

 Again, it does, but you need to be careful for a number of reasons:

i) There may be different sort orders in the new database platform - particularly if you are moving from a mainframe to unix environment (EBCDIC to ASCII)

ii) Default field lengths may be different

iii) Character sets may be different (specifically if you are dealing with non-standard character sets)

iv) Performance tuning starts from scratch again, as different requirements need to be taken into consideration.

v) and many more besides 

Gen can help in a number of ways, but the primary way is that it leaves the DBA-types to concentrate on the tuning needs and results of tuning can be sent to the developers who can concentrate on getting a performant system, and NOT on actually cutting code.

It can help in as much as the focus on a project to change database need not be on the code or the platform-specific stuff needed to get a system up and running, the focus should be on the differences, not the basics.

It could also be argued that Gen hides the database technology from the developer. This is (in the main) a good thing, but thinking about one of my previous posts (Model-based Development and Top Fuel Dragsters)that the Gen developer should understand what goes on under the bonnet - he/she should understand what portions of the application are controlled by his/her code and what parts of the application will be affected by a change in database technology.

Is this abstraction a good or a bad thing ?

It depends on how you manage it.

Giving developers an appreciation of HOW the application and database interact IS valuable, and necessary, but IN-DEPTH knowledge isnt required. Development teams can be stronger with knowledge outside their domain.

Changing Functionality - PTF or not ?

A while ago (we are talking 7 or 8 years) I was involved in construction a portion of a large system which required that a data item be assembled from a number of smaller data items into a long 800 byte string.

Simply speaking, we were concatenating a number of views on to the end of each other to form the resultant export view which we then used. The usage of the result was by a number of third parties (note that this was pretty much before the widespread use of XML for data transmission, so we had to form our own standards which we agreed with the 3rd parties).

The point in all of this was, that after installing an generator PTF to our remote encyclopedia environment as a prerequisite for a fix that we really needed, things started to go awry.

The large 800-byte concatenated view didn’t look right, so we investigated what was wrong.

In the view that we were concatenating into, (defined as variable length), prior to the “fix” it was actually storing trailing spaces from the constituent views, so we didn’t have to worry about padding. Apparently this was incorrect behaviour, so was then fixed by the prerequisite PTF to the generators.

This now caused us a problem, as we had depended on (admittedly) incorrect behaviour previously to allow us to get the results we wanted. NOW, with the PTF applied, the behaviour was corrected and we were stuffed !!!!!

We basically had to rewrite much code to take account of the fact that trailing spaces were no longer stored at the end of a variable legth export view - this took longer than we thought, as analysis showed many more instances of this than we expected.

Move forward to today, and during a large Gen upgrade on the same site, we discovered that things that have been previously working are now not, simply because of “corrections” in new versions of Gen and recodings will be necessary in some cases.

Now I know that the situations described above are not Gen-specific, but what I think would alleviate the situation is better documentation at the outset, so that us developers can see if a function is actually doing what it says in the documentation. In my opinion, no longer is it enough to say that a function “does this”, but it should give detailed examples, illustrate usage, and limitations. This should be similar for events, too as some event stacking priorities have changed over the years.

When things like this change, if they were documented, then people who manage systems can perform proper impact assessments of upgrades.

OK - the situation is better than it used to be, but compare the documentation that comes with (say) VB.Net with Gen documentation - need I say more ?

New set of Gen-related links

 Thanks to a contributor to this blog for some new Gen-related links.

(Short post today!!!)

Particularly this one : ftp://ftp.ca.com/pub/cool/Gen there are many documents of interest here to the Gen user - both casual and committed