Author: MichaelaReydt
Publication Date: 4/3/2013 13:45
In FirstSpirit 4.2 or less it was always necessary to generate a lot of content for a deployment. There was only the choice between a full or a partial generation, so most of the time unchanged content had been generated as well. This caused comprehensive and time-consuming generations. To generate and deploy just the changes since the last generation, you needed to use the revision API to gather the information you needed.
FirstSpirit 5 introduces a new mechanism, called DeltaGeneration, which simplifies things a lot. In the DeltaGeneration a schedule-script searches for only the changed elements since the last successful generation. Especially in large projects this is an advantage, because it reduces the volume of the to be generated elements to a minumum and saves time. (Important) information can be deployed faster.
Note
The DeltaGeneration doesn't work, if the elements inside a project are released automatically. So the checkbox "Use Release function" inside the "Options"-Tab of the Project Properties has to be activated.
This blogposting explains the needed steps for a simple DeltaGeneration in the Mithras-Demoproject:
- Create the Schedule-Task
- Configure the Script
- Configure the Generate-Task
- Try it!
Create the Schedule-Task
After opening the project-properties a new default-schedule can be added inside the schedule-management. The new entry doesn't need a specific name. In this example it is called "Delta-Generation". Furthermore it needs a script and a generate task in the "Actions"-Tab.
Configure the Script
A double-click at the "script"-entry pops up a dialog, in which the following code can be copied.
import de.espirit.firstspirit.access.schedule.*;
deltaGeneration = DeploymentUtil.createDeltaGeneration(context); //(1)
DeltaGeneration.ChangeSet changeSet = deltaGeneration.calculateChangeSet(); //(2)
changeSet.configureGenerateTask(); //(3)
context.logInfo("***** isFullGenerate: " + changeSet.isFullGenerate()); //(4)
At the beginning of this script a new DeltaGeneration is created (1), before the script searches for all changed elements inside the project (2). With the first generate task (after the current script task) this set of changed elements will be generated (3). The log-info at the end of the script (4) is just an information, if the changeset results a full-generation.
Configure the Generation-Task
For the DeltaGeneration the generate task has to be defined as a FullGeneration.
If the checkbox "Generate only if necessary" isn't activated and no elements were changed, the last generated elements will be generated another time.
Else if the checkbox is activated and no changes were made, the generation will be aborted and a warning appears in the logfile:
WARN 20.03.2013 10:45:59.870 (de.espirit.firstspirit.server.scheduler.ScheduleManagerImpl): schedule entry 'Delta-Generation' (id=23909) cancelled - no generation necessary, no changes in project 'Mithras-Project' (id=23940) since last generation, last change was 'Wed Mar 20 10:27:44 CET 2013', revision 20534
Furthermore the "Template sets" has to be activated in the "Extended"-Tab.
Try it!
Now all preparations are done to start a DeltaGeneration. Therefore an element inside the project should be changed and released. The schedule entry can be executed via the main-menu "Project / Execute schedule-entry". After the information dialog that the server generates pages, appears the "Schedule entry status"-dialog. A click at "Detail" opens the logfiles of the schedule entry.
At the end of the script-logfile the information of the last script-line can be found.
line of the script (see above):
context.logInfo("***** isFullGenerate: " + changeSet.isFullGenerate());
information in the script-logfile:
INFO 20.03.2013 13:01:29.430 (de.espirit.firstspirit.impl.access.ScriptContextImpl): ***** isFullGenerate: false
So the server didn't make a FullGeneration although the generate task was configured this way (see image above).
The generate-logfile shows how many elementes were generated.
INFO 20.03.2013 13:01:30.284 (de.espirit.firstspirit.server.scheduler.GenerateTaskExecutor): 2 pages produced, ~384 ms per page
Further Information
This blogposting explains just a simple example with only a few basic-features of a DeltaGeneration. But the API comes up with more methods.
For just a few of them (probably the most important) follows a brief description.
Dependencies
Inside a project elements have depencies to other elements. If also the dependencies of changed elements should be regarded by the generation, dependencyRules are needed.
As long as no dependencyRules are added to the generation-script (as in the given example above) the default dependencyRules are regarded:
- PROPAGATE_PAGE_CHANGES - referenced sitestore-nodes will be generated, if a page has been changed
- PROPAGATE_SECTION_CHANGES - referenced pages will be generated, if a section has been changed
- PROPAGATE_MEDIA_CHANGES_TO_PAGESTORE - referenced pagestore-content will be generated, if a picture or file has been changed
- PROPAGATE_MEDIA_CHANGES_TO_SITESTORE - referenced sitestore-content will be generated, if a picture or file has been changed
- PROPAGETE_TEMPLATE_CHANGES - pagestore-content based on a template will be generated, if this template has been changed
Note
Format templates, which are referenced within a page or section via CMS_RENDER(), need to be defined as type "section".
Else changes made to these format templates will not be identified and the dependencyRule PROPAGATE_TEMPLATE_CHANGES does not have an effect.In this case the related page or section will not be generated.
The dependencyRules could be changed and defined individually by overwriting the default dependencyRules. Therefore a set of the corresponding dependencyRules has to be added to the script in the line before the changeSet is defined:
deltaGeneration = DeploymentUtil.createDeltaGeneration(context);
deltaGeneration.dependencyRules(EnumSet.of(DependencyRule.PROPAGATE_PAGE_CHANGES,
DependencyRule.PROPAGATE_SECTION_CHANGES,
DependencyRule.PROPAGATE_TEMPLATE_CHANGES, // references pagestore-content based on the changed template will be generated
DependencyRule.PROPAGATE_GCA_CHANGES)); // all pages rendering the changed GCA will be generated
DeltaGeneration.ChangeSet changeSet = deltaGeneration.calculateChangeSet();
As soon as individual dependencyRules are defined inside the generation-script, the default dependecyRules will be overwritten and won't be regarded anymore, if they are not added to the set as well.
Compared with the default dependencyRules the rules for page and section changes were kept, but the rules for changed media have been omitted in this example. Instead the rules for changed templates and GCAs were added.
This code is just an example. The API supplies further dependencyRules, which can be combined as needed.
Navigations
In general projects contain navigations with several levels. The image below shows that the Mithras-demoprojects has two navigations with the levels 0 to 1 and 2 to 4.
Changing the structure of the navigation always has an impact on the underlying pages. If a menu item is added to or deleted from a level all pages at the same level or above has to be generated. Otherwise they will keep the old unchanged navigation.
To regard such navigation-changes the levelRules has to be added to the script by changing the definiton of the changeSet.
Instead of:
DeltaGeneration.ChangeSet changeSet = deltaGeneration.calculateChangeSet();
the line might look like this:
DeltaGeneration.ChangeSet changeSet = deltaGeneration.levelRule(0,1).levelRule(2,4).calculateChangeSet();
The numbers in the brackets show the upper and lower limit.
That means in this case:
- If the navigation is changed on level 0 or 1 the generation of the navigation has to start at level 0.
- If the navigation is changed on level 2, 3 or 4 the generation of the navigation has to start at level 2.
Generated / Deleted Files
It is possible to get a list of all generated or deleted files.
This aspect will be explained in the next blogposting. Look forward!
API-Classes
The classes, which are needed for a DeltaGeneration, can be found inside the Developer-API.
Their names are:
Kommentare
8 Kommentare
I tried the example with the levelRule but got an Exception in the calculate calculateChangeSet
Caused by: java.lang.NullPointerException
at de.espirit.firstspirit.access.schedule.ChangeSetCalculator.scanRevisions(ChangeSetCalculator.java:195)
at de.espirit.firstspirit.access.schedule.ChangeSetCalculator.calculateChangeSet(ChangeSetCalculator.java:144)
at de.espirit.firstspirit.access.schedule.DeltaGenerationImpl.calculateChangeSet(DeltaGenerationImpl.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
This is the script code:
This Exception will be fixed in the next release (the internal id of this issue is #133075).
#133075 is fixed in release 5.0.318 and newer.
Hi,
I try to configure a schedule task for a remote project to generate only changed media files. My configuration looks like:
deltaGeneration.dependencyRules(EnumSet.of(DependencyRule.CHANGED_MEDIA));
The API decribes: "Include all changed media nodes. This should be used for projects which serve as media pool for other projects to ensurce that all changed media are generated."
I expected to receive only generated media files. But if the project contains also structure sites, which has been changed and released, they will be also generated within this schedule task.
There is currently no "exclude" for page changes. To achieve this you could manually remove all "non media start nodes" from the start node list.
Lets say we configured a delta deployment and after a successful generation we have a deployment like ftp or similar which fails.
When I perform the next delta deployment what changes will be deployed. Only the changes which has been made since the last failed deployment oder the one before?
@daniel.philippi: The answer is: "Since the last failed deployment".
You can use the variable DeltaGeneration.DELTA_GENERATION_LAST_EXECUTION to read and set the revision id of the next starting point for the delta generation to change this behaviour (ScheduleContext.setVariable / ScheduleContext.getVariable).
Hallo,
ich erhalte einen Fatal Error wenn das PrepareDelta-Skript kein Delta errechnen kann:
INFO 28.07.2016 09:12:50.010 (de.espirit.firstspirit.server.scheduler.ScheduleManagerImpl): starting task 'generate' - schedule entry 'Delta-Generation Master' (id=995551)
ERROR 28.07.2016 09:12:50.014 (de.espirit.firstspirit.server.scheduler.ScheduleManagerImpl): failure running task 'generate' from schedule entry 'Delta-Generation Master' (id=995551) - java.util.concurrent.ExecutionException: de.espirit.firstspirit.access.schedule.CancelSchedule: no generation necessary, no changes in project 'Corporate' (id=60) since last generation, last change was 'Wed Jul 27 14:30:58 CEST 2016', revision 318917
java.util.concurrent.ExecutionException: de.espirit.firstspirit.access.schedule.CancelSchedule: no generation necessary, no changes in project 'Corporate' (id=60) since last generation, last change was 'Wed Jul 27 14:30:58 CEST 2016', revision 318917
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at de.espirit.common.util.SuspendableExecutorService$SuspendableFuture.get(SuspendableExecutorService.java:185)
at de.espirit.firstspirit.server.scheduler.ScheduleManagerImpl$ScheduleRunnable.joinTasks(ScheduleManagerImpl.java:1998)
at de.espirit.firstspirit.server.scheduler.ScheduleManagerImpl$ScheduleRunnable.doRun(ScheduleManagerImpl.java:1944)
at de.espirit.firstspirit.server.scheduler.ScheduleManagerImpl$ScheduleRunnable.run(ScheduleManagerImpl.java:1800)
at de.espirit.firstspirit.server.ExecutionManagerImpl$RunnableWrapper.call(ExecutionManagerImpl.java:634)
at de.espirit.firstspirit.server.ExecutionManagerImpl$ExtendedCallable.call(ExecutionManagerImpl.java:600)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at de.espirit.common.util.BoundedExecutorService$RunnableWrapper.run(BoundedExecutorService.java:436)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at de.espirit.common.util.SuspendableThread.run(SuspendableThread.java:55)
Caused by: de.espirit.firstspirit.access.schedule.CancelSchedule: no generation necessary, no changes in project 'Corporate' (id=60) since last generation, last change was 'Wed Jul 27 14:30:58 CEST 2016', revision 318917
at de.espirit.firstspirit.server.scheduler.GenerateTaskExecutor.run(GenerateTaskExecutor.java:104)
at de.espirit.firstspirit.server.scheduler.ScheduleManagerImpl$TaskCallable.executeLocal(ScheduleManagerImpl.java:2258)
at de.espirit.firstspirit.server.scheduler.ScheduleManagerImpl$TaskCallable.executeLocal(ScheduleManagerImpl.java:2241)
at de.espirit.firstspirit.server.scheduler.ScheduleManagerImpl$TaskCallable.call(ScheduleManagerImpl.java:2164)
Hier der Code dazu:
Wie kann ich den denn umgehen?
Gruß und Danke
Markus
Bitte melden Sie sich an, um einen Kommentar zu hinterlassen.