Iterating across every page - Heap space problems

Zendesk API User

• 7. Dezember 2016 13:52

Author: liam_davison
Publication Date: 12/7/2016 13:52

Good afternoon,

FirstSpirit version 5.1.605.72781.

I am writing a module which will run on a schedule. The module checks each page in turn, checking for certain values like release status, template and so on. For every page that matches, it will be added to a list. The list of pages will then be emailed to some staff members (not CMS users).

I've written my first version of the code, but it always runs out of Heap space when I run it. I suspect that I've written inefficent code - I think this line is at fault:

Iterator<Page> pageIterator = pageStore.getChildren(Page.class, true).iterator();

Is there a better way of iterating over every page on the website?

I have attached the complete Java code. It's unlikely that I will be able to increase the memory or heap space on the server. There are about 1,500 pages on the website. The schedule is likely to run once or twice a month, so it doesn't need to run quickly.

Many thanks,

Liam Davison

Tags: iterate, performance

Verknüpfung mit

• Developers

0

• 

  Zendesk API User

  • 8. Dezember 2016 07:35

  Author: bIT_sosswald - 12/8/2016 7:35
  

  Hi Liam,

  the result of pageStore.getChildren(Page.class, true); is of type Listable<Page> which is a subclass of Iterable. You can iterate directly over that result and there is no need to get an iterator object first.

  Try it this way:

  ...
  Listable<Page> pages = pageStore.getChildren(Page.class, true);
  int topicPageCount = 0;
  
  Map<ContentEditor, List<Page>> pagesPerEditor = new HashMap<ContentEditor, List<Page>>();
  
  for (Page page : pages) {
  [add some magic here]
  }
  ...

  The documentation for the class Iterable says that you should not to "toList()" or something but instead use the iterable directly. See: http://www.e-spirit.com/odfs52/access/de/espirit/common/util/Listable.html#toList()

  Also try to just hold the data you need in the memory and minimize the usage of FirstSpirit objects and the number of times you access e.g. a store! This might decrease your memory usage.

  For example try something like this:

  public class PageEntity {
      private String refName;
      private String displayName;
  
      public PageEntity(Page page, Language language) {
          refName = page.getReferenceName();
          displayName = page.getDisplayName(language);
      }
      public String getRefName() {
          return refName;
      }
      public String getDisplayName() {
          return displayName;
      }
      public void setDisplayName(String displayName) {
          this.displayName = displayName;
      }
      public void setRefName(String refName) {
          this.refName = refName;
      }
  }

  Map<ContentEditor, List<PageEntity>> pagesPerEditor = new HashMap<>();
  ...
  pagesPerEditor.get(ce).add(new PageEntity(page, language));
  ...
  for (PageEntity p : pagesPerEditor.get(ce)) {
      String url = getPageUrl(p);
  
      context.logError("\t" + p.getRefName() + ", " + p.getDisplayName() + ", " + url);
  }

  instead of this:

  Map<ContentEditor, List<Page>> pagesPerEditor = new HashMap<ContentEditor, List<Page>>();
  ...
  pagesPerEditor.get(ce).add(page);
  ...
  for (Page p : pagesPerEditor.get(ce)) {
      String url = getPageUrl(p);
  
      context.logError("\t" + p.getReferenceName() + ", " + p.getDisplayName(language) + ", " + url);
  }

  Maybe this will solve your problem.

  Greetings

  Sandro

  0

Bitte melden Sie sich an, um einen Kommentar zu hinterlassen.