[Community] PrimaGIS 0.6-pre Performance

Ludwig Max Brinckmann ludwigbrinckmann at gmail.com
Sun Jul 16 22:53:27 EEST 2006 

I am just playing with this, so this is the current state.

I have two almost identical setups on a newly installed machine.

   - same machine
   - same Zope/Plone (with the exceptions below)
   - same software: geos 2.2.3, mapserver 4.8.3, postgres 8.1.4, postgis
   1.1.3, proj 4.4.9...
   - same data: remote WMS servers such as Nasa, local mapserver based
   WMS for topography, postgis for weather, shapefiles for some other data

Different are:
Fast Version: PCL 0.10.0 release, GeoAwObj 0.2, ZCO 073, Primagis 0.5x
Slow Version PCL, ZCO, GeoawareObjects, Primagis: all current SVN heads
taken three or four days ago.

My site is also pretty much identical, there might be minor differences in
the rules or so, but nothing that I see having an impact.

 and I have just been timing the rendering of 'individual' layers - this is
not entirely true, as I always render the blue border from a shapefile. The
timing is a bit crude (just done looking at the clock), but we are not
talking milliseconds here:


Rendering of the Nasa Mosaic WMS for my Yunnan site:
Fast: about 10s for the Yunnan overview - the Nasa servers can be slow.
Slow: more than 25 seconds.

Human foot print - a ciesin WMS server
Fast: almost instantaneus
Slow: 20s

Local Topography WMS Server
Fast 25 Seconds
Slow 40 seconds

National Parks Shapefile
Fast 1-2 seconds
Slow 22 seconds

Weather - Local Postgis store
Fast 2 Seconds
Slow 30 seconds


Most of the time is actually spent rendering the maps (or maybe in some of
the calls made when creating the parameters -- I will have a look into this
now)

I just added a few lines to the logs, this is typical:

2006-07-16T21:47:32 INFO PrimaGIS
/var/opt/zope/nextgeneration/Products/PrimaGIS/PrimaGISMap.py::view(), line
372
Starting rendering
------
2006-07-16T21:47:56 INFO PrimaGIS
/var/opt/zope/nextgeneration/Products/PrimaGIS/PrimaGISMap.py::view(), line
383
Finished rendering

(This is where the log entries are:
            log("Starting rendering")
            # Render the map image
            maprenderer = self.getMapRenderer()
            image = maprenderer.render(layers = mapsession.getActiveLayers
(),
                                       size = mapsession.getSize(),
                                       bbox = mapsession.getView
().totuple(),
                                       srs = self.getSpatialReferenceCode(),
                                       units = self.getMapUnits(),
                                       format = self.getOutputFormat(),
                                       bgcolor = self.getBackgroundColor()
or '#ffffff',
                                       transparent = not(
self.getBackgroundColor()))
            log("Finished rendering")

)


On 7/16/06, Sean Gillies <sgillies at frii.com> wrote:
>
> On Jul 13, 2006, at 11:47 AM, Ludwig Max Brinckmann wrote:
>
> > I am just in the process of a complete new installation of the
> > current head revisions of Primagis, ZCO, PCL, GeoAwareobjects,
> > mapserver (4-8-3), geos 2-2-3...
> >
> > Kai has quite apparently put a lot of effort in and there are a
> > number of new features (some of which I 'requested', so I should
> > not complain).
> >
> > However, I noticed quite a performance hit, which can make my map
> > taking more than one minute to render. I first thought it had to do
> > with the requirements of the additional features, but I think the
> > issue is something else.
> >
> > What I was seeing is the following warning message from Zope:
> >
> >  2006-07-13T18:17:39 INFO ZODB.Conflict database conflict error
> > (oid 0x01e8c5, class Products.PrimaGIS.PrimaGISLayer.PrimaGISLayer)
> > at /VirtualHostBase/http/www.yunnantest.com:80/YunnanExplorer/
> > VirtualHostRoot//map/view (8 conflicts, of which 0 were unresolved,
> > since startup at Thu Jul 13 17:58:33 2006)
> > ------
> > 2006-07-13T18:17:56 INFO ZODB.Conflict database conflict error (oid
> > 0x01e8c5, class Products.PrimaGIS.PrimaGISLayer.PrimaGISLayer) at /
> > VirtualHostBase/http/www.yunnantest.com:80/YunnanExplorer/
> > VirtualHostRoot//map/view (9 conflicts, of which 0 were unresolved,
> > since startup at Thu Jul 13 17:58:33 2006)
> >
> > I do not know what the real reason for this is, but two
> > transactions are accessing the same object (the Primagislayer to be
> > rendered) at the same time. What happens next, I believe, is that
> > one of these transactions is then aborted, forcing it to restart. I
> > put a few more log messages into my PrimaGISMap Code and then saw
> > the following:
> >
> > 2006-07-13T19:15:44 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::action(), line 493
> > ZOOM AREA
> > ------
> > 2006-07-13T19:15:44 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::action(), line 497
> > finished
> > ------
> > 2006-07-13T19:15:44 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::getImageMapData(), line 716
> > Getting image map data
> > ------
> > 2006-07-13T19:15:44 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::getImageMapData(), line 753
> > Done imagemapdata
> > ------
> > 2006-07-13T19:15:44 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::view(), line 325
> > Starting rendering
> > ------
> > 2006-07-13T19:15:44 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::view(), line 325
> > Starting rendering
> > ------
> > 2006-07-13T19:15:46 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::getLegendData(), line 778
> > Starting legend processing
> > ------
> > 2006-07-13T19:15:48 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::getLegendData(), line 825
> > Finished legend processing
> > ------
> > 2006-07-13T19:16:12 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::view(), line 336
> > Finished rendering
> > ------
> > 2006-07-13T19:16:12 INFO ZODB.Conflict database conflict error (oid
> > 0x01e8c5, class Products.PrimaGIS.PrimaGISLayer.PrimaGISLayer) at /
> > YunnanExplorer/map/view (27 co
> > nflicts, of which 0 were unresolved, since startup at Thu Jul 13
> > 17:58:33 2006)
> > ------
> > 2006-07-13T19:16:13 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::view(), line 325
> > Starting rendering
> > ------
> > 2006-07-13T19:16:21 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::view(), line 336
> > Finished rendering
> > ------
> > 2006-07-13T19:16:22 INFO ZODB.Conflict database conflict error (oid
> > 0x01e8c5, class Products.PrimaGIS.PrimaGISLayer.PrimaGISLayer) at /
> > YunnanExplorer/map/view (28 co
> > nflicts, of which 0 were unresolved, since startup at Thu Jul 13
> > 17:58:33 2006)
> > ------
> > 2006-07-13T19:16:23 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::view(), line 325
> > Starting rendering
> > ------
> > 2006-07-13T19:16:39 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::view(), line 336
> > Finished rendering
> > ------
> > 2006-07-13T19:16:48 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::view(), line 336
> > Finished rendering
> > ------
> > 2006-07-13T19:16:48 INFO ZODB.Conflict database conflict error (oid
> > 0x01e8c3, class Products.PrimaGIS.PrimaGISLayer.PrimaGISLayer) at /
> > YunnanExplorer/map/view (29 co
> > nflicts, of which 0 were unresolved, since startup at Thu Jul 13
> > 17:58:33 2006)
> > ------
> > 2006-07-13T19:16:49 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::view(), line 325
> > Starting rendering
> > ------
> > 2006-07-13T19:17:03 INFO PrimaGIS /var/opt/zope/default/Products/
> > PrimaGIS/PrimaGISMap.py::view(), line 336
> > Finished rendering
> >
> > This is the output from a single zoom operation on the map. The
> > course of action has an element of non-determinism to it, so it is
> > a timing issue.
> > So, whenever a conflict occurs the operation is reset and restarted
> > (possibly after a delay). The cumulation of these resets makes it
> > all very slow.
> >
> >
> > Does anyone who knows more about these things consider this an
> > issue with my particular installation?
> >
> > Or is this a synchronisation issue, that could be solved with some
> > form of mutex somewhere?
> >
> >
> >
> > Ludwig
>
> Do you happen to have the same data stores and PrimaGISLayers still
> active with an older PrimaGIS installation? What kind of data do you
> have behind the PrimaGISLayer?
>
> Sean
>
> ---
> Sean Gillies
> http://zcologia.com
>
>
>
> _______________________________________________
> Community mailing list
> Community at lists.gispython.org
> http://lists.gispython.org/mailman/listinfo/community
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gispython.org/pipermail/community/attachments/20060716/60a1274f/attachment.htm>

More information about the Community mailing list