Welcome to Planet OSGeo

January 19, 2022

What I love the most in Python is its readability and ease of use, which makes Python a good language to start in programming. To make Python learning easier, I will list some useful tips/tricks. Please note that it uses Python 3.9 and you must have basic Python knowledge to use it properly.

1. Sorting list of list/tuple/dict

We can use key parameter in sort function to provide sorting criteria. We can do this to sort nested list/tuple based on it's last element. If we don't specify key, it will sort based on the first attribute.

>>> a = [[2, 3, 4], [1, 1, 5], [7, 0, -1]]
>>> sorted(a, key=lambda b: b[-1])
[[7, 0, -1], [2, 3, 4], [1, 1, 5]]

If the list contains dict, just change the key to key=lambda b: b['some_key']. If we want to reverse the sorting, add reverse=True.

>>> sorted
(a, key=lambda b: b[-1], reverse=True)
[[1, 1, 5], [2, 3, 4], [7, 0, -1]

For more complex sorting, provide a callback as the key

2. Multiple assignment

We can assign multiple values in one line. In the example below, we assign 4 values to 3 variables. But as b got asterisk in it, it will get values assigned to it as list.

>>> a, *b, c = 1, 2, 3, 4
>>> a
1
>>> b
[2, 3]
>>> c
4

When the number of values equals to the number of variables, it will be stored as list with one element.

>>> *b = 1
>>> b
[1]

3. Swapping variables

Swapping 2 variables can be easily done with this trick.

>>> a, b = 1, 2
>>> print(a, b)
1, 2
>>> a, b = b, a
>>> print(a, b)
2, 1

4. Combining strings

We can use str.join() to join list of strings, by providing a string to be used as concatenator. See that the string we provide (' and ') is placed between each element, like between Anna and Bob.

>>> persons = ['Zakki', 'Anna', 'Bob', 'Charlie']
>>> ' and '.join(persons)
'Zakki and Anna and Bob and Charlie'

5. Put variables to string

We could use f-string to put variables to string.

>>> subject = 'Zakki'
>>> object = 'window'
>>> f'{subject} is looking at the {object}'
'Zakki is looking at the window'

6. Merge dictionaries

>>> a = {'a': 1}
>>> b = {'b': 2}
>>> d = {**a, **b, **{'c': 3}}
>>> d
{'a': 1, 'b': 2, 'c': 3}

You can also use this (Python 3.9+ only)

>>> d = a | b | {'c': 3}
>>> d
{'a': 1, 'b': 2, 'c': 3}

7. Map elements

Using this trick, we could map an element in one list/tuple to another list/tuple

>>> a = [1, 2, 3]
>>> b = ['a', 'b', 'c']
>>> list(zip(a, b))
[(1, 'a'), (2, 'b'), (3, 'c')]
>>> list(zip(b, a))
[('a', 1), ('b', 2), ('c', 3)]

8. Ternary operator

Use simple this one line ternary operator with syntax value_when_true if condition else value_when_false

>>> a = 'Big'
>>> b = 'Small' if a == 'Big' else 'Big'
>>> b
>>> 'Small'

Instead of

if a == 'Big':
b = 'Small'
else:
b = 'Big'

9. Removing duplicates

By converting to set, we get the unique values.

>>> a = [1, 1, 2, 2, 3]
>>> a
[1, 1, 2, 2, 3]
>>> list(set(a))
[1, 2, 3]

10. Find elements showing in both list

We can do that by converting to set, then use set.intersection()

>>> a = [1, 2, 3]
>>> b = [1, 2]
>>> set_a = set(a)
>>> set_b = set(b)
>>> set_a.intersection(set_b)
{1, 2}

I hope this article helps you in many ways.

by Zulfikar Akbar Muzakki at January 19, 2022 07:27 AM

January 18, 2022

A common spatial need is to find a polygon that accurately represents a set of points.  The convex hull of the points often does not provide this, since it can enclose large areas which contain no points.  What is required is a non-convex hull, often termed the concave hull.  

The Convex Hull and a Concave Hull of a point set

A concave hull is generally considered to have some or all of the following properties:

  • The hull is a simply connected polygon
  • It contains all the input points
  • The vertices in the hull polygon boundary are all input points
  • The hull may or may not contain holes
For a typical point set there are many polygons which meet these criteria, with varying degrees of concaveness.  Concave Hull algorithms provide a numeric parameter which controls the amount of concaveness in the result.  The nature of this parameter is particularly important, since it affects the ease-of-use in practical scenarios.  Ideally it has the following characteristics:

  • Simple geometric basis: this allows the user to understand the effect of the parameter and aids in determining an effective value
  • Scale-free (dimensionless): this allows a single parameter value to be effective on varying sizes of geometry, which is essential for batch or automated processing
  • Local (as opposed to global):  A local property (such as edge length) gives the algorithm latitude to determine the concave shape of the points. A global property (such as area) over-constrains the possible result. 
  • Monotonic area:  larger (or smaller) values produce a sequence of more concave areas
  • Monotonic containment :the sequence of hulls produced are topologically nested
  • Convex-bounded: an extremal value produces the convex hull

This is a well-studied problem, and many different approaches have been proposed.  Some notable ones are:

Of these, Delaunay Erosion (Chi-shapes) offers the best set of features.  It is straightforward to code and is performant.  It uses the control parameter of Edge Length Ratio, a fraction of the difference between the longest and shortest edges in the underlying Delaunay triangulation.  This is easy to reason about, since it is scale-free and corresponds to a simple property of the point set (that of distance between vertices).  It can be extended to support holes.  And it has a track record of use, notably in Oracle Spatial.  

ConcaveHull generated by Delaunay Erosion with Edge Length Ratio = 0.3

Recently the Park-Oh algorithm has become popular, thanks to a high-quality implementation in Concaveman project (which has spawned numerous ports).  However, it has some drawbacks.  It can't support holes (and likely not disconnected regions and discarding outlier points).  As the paper points out and experiment confirms, it produces rougher outlines than the Delaunay-based algorithm.  Finally, the control parameter for Delaunay Erosion has a simpler geometrical basis which makes it easier to use.

Given these considerations, the new JTS ConcaveHull algorithm utilizes Delaunay Erosion. The algorithm ensures that the computed hull is simply connected, and contains all the input points.  The Edge Length Ratio is used as the control parameter. A value of 1 produces the convex hull; 0 produces a concave hull of minimal size.  Alternatively the maximum edge length can be specified directly. This allows alternative strategies to determine an appropriate length value; for instance, another possibility is to use a fraction of the longest edge in the Minimum Spanning Tree of the input points.  

The recently-added Tri data structure provides a convenient basis for the implementation,.  It operates as follows:

  1. The Delaunay Triangulation of the input points is computed and represented as a set of of Tris
  2. The Tris on the border of the triangulation are inserted in a priority queue, sorted by longest boundary edge
  3. While the queue is non-empty, the head Tri is popped from the queue.  It is removed from the triangulation if it does not disconnect the area.  Insert new border Tris into the queue if they have a boundary edge length than the target length
  4. The Tris left in the triangulation form the area of the Concave Hull 

Thanks to the efficiency of the JTS Delaunay Triangulation the implementation is quite performant, approaching the performance of a Java port of Concaveman.  

Concave Hull of Ukraine dataset; Edge Length Ratio = 0.1

Optionally holes can be allowed to be present in the hull polygon (while maintaining a simply connected result).  A classic demonstration of this is to generate hulls for text font glyphs:


This algorithm is in the process of being ported to GEOS.  The intention is to use it to enhance the PostGIS ST_ConcaveHull function, which has known issues and has proven difficult to use.

Further Ideas

  • Disconnected Result - It is straightforward to extend the algorithm to allow a disconnected result (i.e. a MultiPolygon).  This could be provided as an option.
  • Outlier Points - It is also straightforward to support discarding "outlier" points.
  • Polygon Concave Hull - computing a concave "outer hull" for a polygon can be used to simplify the polygon while guaranteeing the hull contains the original polygon.  Additionally, an "inner hull" can be computed which is fully contained in the original.  The implementation of a Polygon Concave Hull algorithm is well under way and will be released in JTS soon. 



by Dr JTS (noreply@blogger.com) at January 18, 2022 02:53 PM

Civil rights activist and Baptist minister Martin Luther King, Jr. was born on January 15, 1929 and was assassinated on April 4, 1968. Today was the official US holiday honoring King and his legacy, the 35th year we've done so in Colorado.

White Americans are highly selective in how they remember and honor King. Many of us think only of the following, pat ourselves on the back while thinking of the non-white musicians, comedians, athletes, celebrities who entertain us, and consider King's work done.

I look to a day when people will not be judged by the color of their skin, but by the content of their character.

Martin Luther King, Jr. was an anti-racist, anti-capitalist, and anti-militarist, and was aiming beyond today's state of color-blindness for the well off, towards a truly equitable world. One where your family's wealth didn't depend on your skin color. One where workers were fairly compensated for their labor and treated with respect. One where education was prioritized over jets, bombs, drones, and extravagant military adventures. I implore you to read up on this and push back on people who try to whitewash King's legacy.

by Sean Gillies at January 18, 2022 01:50 AM

2021 was my biggest year of running. I'm reigning myself in a bit for 2022, targeting only a 100 kilometer run at the end of August or a 50 miler at the beginning of September, with one other 50-60k or trail marathon as a training run during summer. I'm going to do plenty of running, but no serious workouts until later in March. Until then I will be prioritizing skiing with family and friends.

https://live.staticflickr.com/65535/51828357620_3fde054dcc_b.jpg

New Years Day 2022

by Sean Gillies at January 18, 2022 01:31 AM

January 17, 2022

The VirES for Aeolus service has been created for the purpose of helping the public as well as scientists access and analyze the data from ESA's Aeolus mission. We are happy to see that the capabilities of the service to quickly browse through the mission data are being used to find events captured ...

January 17, 2022 12:00 AM

January 15, 2022

The latest v0.9 release is now available from conda-forge.

This release contains some really cool new algorithms:

The Kalman filter in action on the Geolife sample: smoother, less jiggly trajectories.
Top-Down Time Ratio generalization aka trajectory compression in action: reduces the number of positions along the trajectory without altering the spatiotemporal properties, such as speed, too much.

These new algorithms were contributed by Lyudmil Vladimirov and George S. Theodoropoulos.

Behind the scenes, Ray Bell took care of moving testing from Travis to Github Actions, and together we worked through the steps to ensure that the source code is now properly linted using flake8 and black.

Being able to work with so many awesome contributors has made this release really special for me. It’s great to see the project attracting more developer interest.

As always, all tutorials are available from the movingpandas-examples repository and on MyBinder:

by underdark at January 15, 2022 06:04 PM

January 14, 2022

As of PostGIS 3.0, the PostGIS raster support is no longer part of the postgis extension, but instead spun off into a new PostGIS extension called postgis_raster.

The two main reasons for this break were:

  • Raster functionality in PostGIS is fat with over 150 functions and several types. Wading through these extra functions frustrated many who had no use for rasters.

  • Raster gdal dependency is very big and many dreamed of having postgis extension without the big raster dependency.

While repackaging raster as it’s own extension resolved many complaints, it meant a slightly more complicated upgrade process going from PostGIS 2.something to 3.something that even experienced PostGIS users managed to screw up.

I will detail the proper way of upgrading PostGIS raster, from a PostGIS 2.* install to 3.* install.

You can run these steps using psql or pgAdmin or any other PostgreSQL tool you want.

Regardless which version of PostGIS you are coming from, you should install the PostGIS 3.* binaries first.

Continue Reading by clicking title hyperlink ..

by Regina Obe at January 14, 2022 12:00 AM

January 13, 2022

Les extensions QGIS Serveur et comment les gérer

Extensions QGIS

Que serait QGIS sans ses extensions ? C'est surement la première chose que l'on fait une fois que l'on vient d'installer QGIS sur un nouvel ordinateur. On ouvre le gestionnaire des extensions et on installe nos extensions favorites.

QGIS Bureautique fournit une interface graphique intuitive pour gérer ses extensions. Vous pouvez facilement les lister, les installer ou les mettre à jour.

Astuce rapide, dans votre gestionnaire, onglet "Paramètres", n'oubliez pas de cocher la case Chercher des mises à jour au démarrage au moins une fois par mois. Cela serait triste de ne pas profiter des dernières fonctionnalités ou alors de continuer à voir le même bug alors qu'il est déjà corrigé dans une version récente.

Gestionnaire extension QGIS bureautique

Extension QGIS Serveur

Peut-être que vous ne le savez pas, mais vous pouvez ajouter des extensions sur QGIS Serveur également. Vous pouvez trouver la liste des extensions compatibles pour le serveur sur plugins.qgis.org. Une extension sur QGIS Serveur peut-être utile pour ajouter des nouvelles fonctionnalités ou alors modifier le fonctionnement par défaut. Regardons l'exemple concret de l'extension ci-dessous.

Un cas pratique, le cas de WfsOutputExtension

Le WFS est un standard défini par l'Open Geospatial Consortium afin de permettre aux utilisateurs de télécharger des données vectorielles de QGIS Serveur dans un client OGC tel que QGIS Bureautique, ArcGIS ou tout autre application Web-SIG (disonsLizmap Web Client par hasard). Par défaut, ce standard permet le téléchargement des données au format GML uniquement. Cependant, nos clients souhaitent pour la plupart télécharger dans des formats plus communs comme un shapefile compressé, un geopackage, un fichier ODS ou CSV.

C'est pourquoi à 3Liz, nous avons développé l'extension serveur WfsOutputExtension.

Cette extension ajoute des formats d'exports additionnels à la volée. En arrière-plan, il intercepte la requête GetCapabilities WFS et ajoute les formats supplémentaires dans la réponse.

Du côté du client OGC, l'utilisateur peut choisir un de ces formats lors de la demande des données vectorielles. C'est visible par exemple dans l'interface du client web Lizmap :

WfsOutputExtension in action

QGIS-Plugin-Manager

Comme dit plus haut, QGIS Bureautique permet, à l'aide d'une interface graphique intuitive, de gérer les extensions du dépôt principal QGIS. Mais quand vous êtes sur un serveur, il n'y a pas d'outil pour installer ou mettre à jour les extensions. À 3Liz, pour gérer notre infrastructure, nous utilisons des outils de déploiement comme Ansible pour installer les extensions.

Mais nous déployons aussi QGIS Serveur et Lizmap Web CLient sur les serveurs de nos clients, en dehors de notre infrastructure. Cela signifie qu'ils hébergent leur propre serveur et qu'ils nous demandent de faire l'installation et la maintenance de ce dernier. Dans ce contexte, nous installions au début les extensions QGIS Serveur à l'aide des commandes wget, unzip etc. Ceci n'est pas très pratique ni robuste sur long terme.

C'est pourquoi, j'ai commencé un petit outil Python en ligne de commande pour lister, installer et mettre à jour les extensions dans un environnement sans interface graphique (comme un serveur Linux).

Cet outil peut être installé avec PIP sur un serveur via la commande suivante :

pip3 install qgis-plugin-manager

Ensuite, vous pouvez utiliser la commande qgis-plugin-manager, par exemple pour avoir la liste des extensions depuis une console bash :

$ qgis-plugin-manager list
QGIS server version 3.19.0
List all plugins in /home/etienne/dev/qgis/server_plugin

----------------------------------------------------------------------------------------
|  Name            |  Version  |  QGIS min  |  QGIS max  |  Author         |  Action ⚠       |
----------------------------------------------------------------------------------------
|Lizmap            |master     |3.4         |3.99        |3Liz             |Unkown version   |
|wfsOutputExtension|1.5.3      |3.0         |            |3Liz             |                 |
|QuickOSM          |1.14.0     |3.4         |3.99        |Etienne Trimaille|Upgrade to 1.16.0|
|cadastre          |1.6.2      |3.0         |3.99        |3liz             |                 |
|atlasprint        |3.2.2      |3.10        |            |3Liz             |                 |
----------------------------------------------------------------------------------------

En arrière-plan, cet outil utilise les concepts d'APT et de PIP. Il est connecté par défaut au dépôt principal de QGIS, mais vous pouvez également utiliser votre propre dépôt d'extensions comme celui de 3liz : https://docs.3liz.org/plugins.xml.

Je vous suggère de lire la documentation dans le fichier readme. Ce projet est hébergé sur GitHub donc n'hésitez pas à faire des tickets ou encore mieux un "pull request" si vous voulez contribuer 😉.

by Etienne at January 13, 2022 08:00 AM

QGIS Server plugins and how to manage them all

QGIS plugins

What would be QGIS without plugins ? It might be the first thing we do when we install QGIS on a new computer. We open the QGIS plugin manager, and we install our favorite plugins.

QGIS Desktop is providing a nice interface to manage plugins. You can easily list, install or upgrade them.

Quick important tip, in your plugin manager, in the Settings tab, do not forget to enable the Check for updates on startup at least every month. QGIS Plugin developers are releasing new versions of their plugins very often, it would be sad to miss new features or to keep having the same bug while it's already fixed !

QGIS Plugin manager in desktop

QGIS Server plugins

Maybe you don't know, but you can install plugins on QGIS Server. You can find the list of plugins compatible for QGIS Server on plugins.qgis.org. Plugins on QGIS Server can be useful to add new features or to alter the default behavior of QGIS Server. Let's have a look to a specific plugin below.

A use case, the WfsOutputExtension plugin

WFS is a standard made by the Open Geospatial Consortium to allow users to download vector data from QGIS Server in a OGC client such as QGIS Desktop, ArcGIS or any Web-GIS application (let's say randomly Lizmap Web Client). By default, this standard allows only the data to be downloaded as GML format. However, our customers wanted to download vector format with a more friendly vector format such as zipped shapefile, geopackage, ODS or CSV. Therefore, at 3Liz, we have developed the WfsOutputExtension server plugin.

This plugin adds additional export formats on the fly. In the background, the plugin intercepts the WFS GetCapabilities request and adds additional formats in the XML response. From the OGC client side, the user can use these formats when requesting vector data. It's visible from Lizmap Web Client in the dropdown menu:

WfsOutputExtension in action

QGIS-Plugin-Manager

As I mentioned, QGIS Desktop provides a nice way to manage plugins which are listed on the main QGIS repository. But when you are on a server, there isn't such tool to install and upgrade plugins. At 3Liz, we are managing our own server infrastructure. Deploying and upgrading plugins is done using some deployments tools such as Ansible.

We also deploy QGIS Server and Lizmap Web Client on our customers servers, outside of our hosting infrastructure. It means they host their own server and they ask us to do some maintenance on it. In this context, we previously installed QGIS Server plugins manually with the help of some wget and unzip commands. Not very convenient in the long term…

Therefore, I started a small Command Line Interface with Python to list and upgrade plugins when you are on a headless environment.

This tool can be installed with PIP on a server:

pip3 install qgis-plugin-manager

Then, you can list and install plugins from the bash prompt:

$ qgis-plugin-manager list
QGIS server version 3.19.0
List all plugins in /home/etienne/dev/qgis/server_plugin

----------------------------------------------------------------------------------------
|  Name            |  Version  |  QGIS min  |  QGIS max  |  Author         |  Action ⚠       |
----------------------------------------------------------------------------------------
|Lizmap            |master     |3.4         |3.99        |3Liz             |Unkown version   |
|wfsOutputExtension|1.5.3      |3.0         |            |3Liz             |                 |
|QuickOSM          |1.14.0     |3.4         |3.99        |Etienne Trimaille|Upgrade to 1.16.0|
|cadastre          |1.6.2      |3.0         |3.99        |3liz             |                 |
|atlasprint        |3.2.2      |3.10        |            |3Liz             |                 |
----------------------------------------------------------------------------------------

In the background, this tool is using ideas from APT and PIP. It's connected to the main QGIS plugin repository, but you can also add our own plugin repository like https://docs.3liz.org/plugins.xml

I suggest you to read the full documentation in the readme file. This project is hosted on GitHub so feel free to raise issues and even better, to make some pull requests 😉

by Etienne at January 13, 2022 08:00 AM

January 12, 2022

Les extensions QGIS Serveur et comment les gérer

Extensions QGIS

Que serait QGIS sans ses extensions ? C'est surement la première chose que l'on fait une fois que l'on vient d'installer QGIS sur un nouvel ordinateur. On ouvre le gestionnaire des extensions et on installe nos extensions favorites.

QGIS Bureautique fournit une interface graphique intuitive pour gérer ses extensions. Vous pouvez facilement les lister, les installer ou les mettre à jour.

Astuce rapide, dans votre gestionnaire, onglet "Paramètres", n'oubliez pas de cocher la case Chercher des mises à jour au démarrage au moins une fois par mois. Cela serait triste de ne pas profiter des dernières fonctionnalités ou alors de continuer à voir le même bug alors qu'il est déjà corrigé dans une version récente.

Gestionnaire extension QGIS bureautique

Extension QGIS Serveur

Peut-être que vous ne le savez pas, mais vous pouvez ajouter des extensions sur QGIS Serveur également. Vous pouvez trouver la liste des extensions compatibles pour le serveur sur plugins.qgis.org. Une extension sur QGIS Serveur peut-être utile pour ajouter des nouvelles fonctionnalités ou alors modifier le fonctionnement par défaut. Regardons l'exemple concret de l'extension ci-dessous.

Un cas pratique, le cas de WfsOutputExtension

Le WFS est un standard défini par l'Open Geospatial Consortium afin de permettre aux utilisateurs de télécharger des données vectorielles de QGIS Serveur dans un client OGC tel que QGIS Bureautique, ArcGIS ou tout autre application Web-SIG (disonsLizmap Web Client par hasard). Par défaut, ce standard permet le téléchargement des données au format GML uniquement. Cependant, nos clients souhaitent pour la plupart télécharger dans des formats plus communs comme un shapefile compressé, un geopackage, un fichier ODS ou CSV.

C'est pourquoi à 3Liz, nous avons développé l'extension serveur WfsOutputExtension.

Cette extension ajoute des formats d'exports additionnels à la volée. En arrière-plan, il intercepte la requête GetCapabilities WFS et ajoute les formats supplémentaires dans la réponse.

Du côté du client OGC, l'utilisateur peut choisir un de ces formats lors de la demande des données vectorielles. C'est visible par exemple dans l'interface du client web Lizmap :

WfsOutputExtension en action

QGIS-Plugin-Manager

Comme dit plus haut, QGIS Bureautique permet, à l'aide d'une interface graphique intuitive, de gérer les extensions du dépôt principal QGIS. Mais quand vous êtes sur un serveur, il n'y a pas d'outil pour installer ou mettre à jour les extensions. À 3Liz, pour gérer notre infrastructure, nous utilisons des outils de déploiement comme Ansible pour installer les extensions.

Mais nous déployons aussi QGIS Serveur et Lizmap Web CLient sur les serveurs de nos clients, en dehors de notre infrastructure. Cela signifie qu'ils hébergent leur propre serveur et qu'ils nous demandent de faire l'installation et la maintenance de ce dernier. Dans ce contexte, nous installions au début les extensions QGIS Serveur à l'aide des commandes wget, unzip etc. Ceci n'est pas très pratique ni robuste sur long terme.

C'est pourquoi, j'ai commencé un petit outil Python en ligne de commande pour lister, installer et mettre à jour les extensions dans un environnement sans interface graphique (comme un serveur Linux).

Cet outil peut être installé avec PIP sur un serveur via la commande suivante :

pip3 install qgis-plugin-manager

Ensuite, vous pouvez utiliser la commande qgis-plugin-manager, par exemple pour avoir la liste des extensions depuis une console bash :

$ qgis-plugin-manager list
QGIS server version 3.19.0
List all plugins in /home/etienne/dev/qgis/server_plugin

----------------------------------------------------------------------------------------
|  Name            |  Version  |  QGIS min  |  QGIS max  |  Author         |  Action ⚠       |
----------------------------------------------------------------------------------------
|Lizmap            |master     |3.4         |3.99        |3Liz             |Unkown version   |
|wfsOutputExtension|1.5.3      |3.0         |            |3Liz             |                 |
|QuickOSM          |1.14.0     |3.4         |3.99        |Etienne Trimaille|Upgrade to 1.16.0|
|cadastre          |1.6.2      |3.0         |3.99        |3liz             |                 |
|atlasprint        |3.2.2      |3.10        |            |3Liz             |                 |
----------------------------------------------------------------------------------------

En arrière-plan, cet outil utilise les concepts d'APT et de PIP. Il est connecté par défaut au dépôt principal de QGIS, mais vous pouvez également utiliser votre propre dépôt d'extensions comme celui de 3liz : https://docs.3liz.org/plugins.xml.

Je vous suggère de lire la documentation dans le fichier readme. Ce projet est hébergé sur GitHub donc n'hésitez pas à faire des tickets ou encore mieux un "pull request" si vous voulez contribuer 😉.

by Etienne at January 12, 2022 12:00 PM

QGIS Server plugins and how to manage them all

QGIS plugins

What would be QGIS without plugins ? It might be the first thing we do when we install QGIS on a new computer. We open the QGIS plugin manager, and we install our favorite plugins.

QGIS Desktop is providing a nice interface to manage plugins. You can easily list, install or upgrade them.

Quick important tip, in your plugin manager, in the Settings tab, do not forget to enable the Check for updates on startup at least every month. QGIS Plugin developers are releasing new versions of their plugins very often, it would be sad to miss new features or to keep having the same bug while it's already fixed !

QGIS Plugin manager in desktop

QGIS Server plugins

Maybe you don't know, but you can install plugins on QGIS Server. You can find the list of plugins compatible for QGIS Server on plugins.qgis.org. Plugins on QGIS Server can be useful to add new features or to alter the default behavior of QGIS Server. Let's have a look to a specific plugin below.

A use case, the WfsOutputExtension plugin

WFS is a standard made by the Open Geospatial Consortium to allow users to download vector data from QGIS Server in a OGC client such as QGIS Desktop, ArcGIS or any Web-GIS application (let's say randomly Lizmap Web Client). By default, this standard allows only the data to be downloaded as GML format. However, our customers wanted to download vector format with a more friendly vector format such as zipped shapefile, geopackage, ODS or CSV. Therefore, at 3Liz, we have developed the WfsOutputExtension server plugin.

This plugin adds additional export formats on the fly. In the background, the plugin intercepts the WFS GetCapabilities request and adds additional formats in the XML response. From the OGC client side, the user can use these formats when requesting vector data. It's visible from Lizmap Web Client in the dropdown menu:

WfsOutputExtension in action

QGIS-Plugin-Manager

As I mentioned, QGIS Desktop provides a nice way to manage plugins which are listed on the main QGIS repository. But when you are on a server, there isn't such tool to install and upgrade plugins. At 3Liz, we are managing our own server infrastructure. Deploying and upgrading plugins is done using some deployments tools such as Ansible.

We also deploy QGIS Server and Lizmap Web Client on our customers servers, outside of our hosting infrastructure. It means they host their own server and they ask us to do some maintenance on it. In this context, we previously installed QGIS Server plugins manually with the help of some wget and unzip commands. Not very convenient in the long term…

Therefore, I started a small Command Line Interface with Python to list and upgrade plugins when you are on a headless environment.

This tool can be installed with PIP on a server:

pip3 install qgis-plugin-manager

Then, you can list and install plugins from the bash prompt:

$ qgis-plugin-manager list
QGIS server version 3.19.0
List all plugins in /home/etienne/dev/qgis/server_plugin

----------------------------------------------------------------------------------------
|  Name            |  Version  |  QGIS min  |  QGIS max  |  Author         |  Action ⚠       |
----------------------------------------------------------------------------------------
|Lizmap            |master     |3.4         |3.99        |3Liz             |Unkown version   |
|wfsOutputExtension|1.5.3      |3.0         |            |3Liz             |                 |
|QuickOSM          |1.14.0     |3.4         |3.99        |Etienne Trimaille|Upgrade to 1.16.0|
|cadastre          |1.6.2      |3.0         |3.99        |3liz             |                 |
|atlasprint        |3.2.2      |3.10        |            |3Liz             |                 |
----------------------------------------------------------------------------------------

In the background, this tool is using ideas from APT and PIP. It's connected to the main QGIS plugin repository, but you can also add our own plugin repository like https://docs.3liz.org/plugins.xml

I suggest you to read the full documentation in the readme file. This project is hosted on GitHub so feel free to raise issues and even better, to make some pull requests 😉

by Etienne at January 12, 2022 12:00 PM

January 10, 2022

Hi, my name is Sean Gillies, and this is my blog. Blog is short for "web log". Individual posts to a web log are called "posts", not "blogs". I write about running, cooking and eating, travel, family, programming, Python, API design, geographic data formats and protocols, open source, and internet standards. Mostly running and local geography. I live in Fort Collins, Colorado, and sometimes in Montpellier, France. I work at Planet Labs PBC. I appreciate emailed comments on my posts. You can find my address in the "about" page linked at the top of this page. Happy New Year!

by Sean Gillies at January 10, 2022 04:33 AM

I finally got prescription glasses. I've needed them for a couple of years. Like a lot of middle-aged people I am slightly near-sighted. On top of this I have some astigmatism which is especially bad when I haven't slept well. I see pairs of stars at night, pairs of hawks soaring in the distance. It really messes with my ability to read text on a screen from the back of a room at the end of a long day.

https://live.staticflickr.com/65535/51810212262_29aac0fe8f_c.jpg

I'm wearing them while driving, shopping, watching movies (on our 4k TV), and reading text on the external monitor in my office. I don't wear them while running or reading a book before bed. While I'm still getting used to circular things not being exactly circular, I'm enjoying a sharper view of the world beyond my laptop screen.

by Sean Gillies at January 10, 2022 01:11 AM

January 04, 2022

Overview of my professional life in 2021. Highlights of living and working in the Open Source Geospatial and OSGeo(.nl |.org )-world in 2021. Organized by "Theme".

Like 2020, 2021 was again a "special year" in pandemic terms. How to cope? Like Paul Ramsey writes: I feel like sharing some of my favourite things . Will do in a later post: Bird-watching , Wolf-tracking workshop , 'catching' big game on new wildcam, developed outdoor GPS-game (Veluwe Big Five), some progress on guitar-playing, discovering The Netherlands outback, becoming a Billy Strings fan. These are a few of my favourite things, more later....

TL;DR Main three 2021 highlights:

Below follow 2021-highlights organized by theme and/or project.

1. Fun Home Automation Projects

Some of my 2022 fun resolutions resolved in 2021:

More on these later.

2. New Website

For years my website/blog justobjects.nl ran on Wordpress at the famous, and first (est. 1993), Dutch internet/hosting provider XS4ALL . XS4ALL was sold to KPN (1998), remained an independent brand, but was finally dismantled in 2021.

So in early 2021 a new hosting provider was needed. This was also a good moment to migrate my website to Hugo . With the great effort by @peter-govind (via freelance.com ), we converted the complete site (about 20 years of blogs) to Hugo+Github , using GitHub Workflows to "publish-on-commit". My first piece of GitOps, more to follow, in 2021. Though there are still some fixes to do, very happy with the result. Maintaining is now a breeze.

3. GitOps

Gradually GitOps became my main theme in 2021. Discovered that there was actually a name (GitOps) for CI/CD practices I (you) have been doing over the past years. For example, in 2019 I worked on the pygeoapi demo server . GitOps can always be taken further until reaching the phase: "look ma, no hands!". No more "Box Hugging", i.e. no more SSH-ing, manual actions on VMs etc.

So what is GitOps? Many definitions are available. Often tied to Kubernetes (K8s), but GitOps is broader than K8s. The most concise I found at RedHat :

"GitOps is a set of practices to manage infrastructure and application configurations using Git, an open source version control system. GitOps works by using Git as a single source of truth for declarative infrastructure and applications."

I often quote here: "The Truth is in Git!". Especially now in times of fake-news, I love speaking those words in presentations. The quest of humanity is over: We Found The Truth, It Is In Git :-). Pun aside, it takes effort to have 100% of your deployment sources and configs in Git. In particular all required credentials (passwords, keys etc). Think of storing credentials in a public GitHub repo, how? Did some amazing discoveries in that area like (combining) GitHub Secrets and Ansible Vault .

I was very lucky that my main three contracts in 2021 involved GitOps. One project sort of followed the other.

Geonovum

It all began in June at Geonovum who started an OGC API Testbed as part of their ongoing Dutch API Strategy theme.

Together with GeoCat Paul van Genuchten , developed a full-stack deployment for several OGC API Features implementations. The picture below depicts the stack elements:

The challenge was to provide a testbed environment where third parties could further extend and experiment. Eventually we developed a deployment fully based on GitOps within GitHub as a GitHub Template repo. You can read details in the documentation . The picture below depicts the GitOps workflow:

These results were presented at several occasions in 2021. See the Presentations below.

Geocat

For GeoCat (again with Paul van Genuchten), exploring and setting out how to further migrate to full GitOps for the GeoCat Live environment.

European Commission - Joint Research Centre

Was already working on a contract with EC-JRC for pan-EU data sharing as a member of the "EU JRC ELISE Pool of Experts". An honour to be (still) working with all these great minds from the international geospatial world!

My focus within the team was/is on data services deployment. My pitch and first deliverable, was called "The Power of Orchestrated Containerisation for Data Services".

The full outcome of the project will be published (publicly) as a JRC Technical Report (JRC127730?) titled: "Emerging approaches for data-driven innovation in Europe". For this report, wrote the chapter "Deployment of an OGC API Features web service using GitOps"

Apart from reporting in text, also provided a full-stack OGC API Features service based on pygeoapi . Using the Geonovum OGC API GitHub Template on a fresh Ubuntu VM provided by JRC, this was a great Proof-of-Concept: took just 2 hours to get the complete stack, including stuff like SSL-certificates, Traefik proxy (love Traefik!), OS-hardening etc, up and running . For now the GitHub repo is at github.com/justb4/ogc-api-jrc/ .

4. OSGeo.nl

FOSS4G-NL 2021 - IRL

Yes, it happened! In between COVID-Waves a LOC lead by the great Erik Meerburg achieved to organize an in-real-life (IRL) 4th OSGeo.nl FOSS4G-NL conference . Abiding with COVID-regulations at the time. At ITC Enschede on October 19-20 2021. Using 2G access. Zero COVID-cases after.

One day of Workshops followed by a one-day conference with great international keynotes by Amelie A Gagnon from UNESCO-IIEP and Kurt Menke from Septima DK , plus three parallel sessions. We are still preparing the website to publish all presentations.

Looks unreal this group photo, but this was in 2021! Thanks to all who made this event a success!

The Big Geo Show

This was already my 2020 highlight. You can watch back all shows via tv.osgeo.nl or directly on the OSGeo.nl YouTube Channel . Hell, you can even binge-watch all episodes !

In 2021 we did six shows, each a Special:

With a real instant-music-composer-performer, Luciën Greefkes in our team, we together made a new DGGS leader video .

OSGeo.nl - Ten Year Anniversary

OSGeo.nl was established on Dec 2, 2011. So 10 years, still going strong! On dec 2, 2021 we were in a new wave, so a small online event. Below the epic picture (by Arnulf Christl) taken at the GIN Congres Utrecht 2011 where we got together to start OSGeo.nl.

A special episode of the Big Geo Show presents: Ten Years of OSGeo.nl - Story in Pictures .

5. Geospatial Cloud Services

Moved into providing Geospatial Cloud Services last few years, both as a source of income and to support/strengthen underlying open source projects with which they are developed. Warning: shameless ads below. All below was ongoing in 2021.

  • map5.nl is a subscription service for Dutch topographic, historical- and embellished hill-shade and arial maps I started to host in 2015. In 2021 work started for a map5.nl user portal.
  • GeoQoS.com is a hosted GeoHealthCheck (GHC) service on a subscription basis. GHC is an uptime and QoS monitor for (OGC) web services. Customers get their own GHC instance. GeoQoS.com saves the burden of self-hosting GHC. In 2021 started work on a GHC API, using FastAPI , and a new GHC-UI.
  • geotoko.nl , in short GeoToko, is basically a webshop to download Dutch open geo-datasets. One may ask: Dutch geospatial data like Topography, Addresses and Buildings, is already open and publicly available, mainly via Kadaster-PDOK . So why bother reselling? Explanation follows.

Most Dutch Open geospatial datasets, available from governmental institutions like Kadaster, are provided in a neutral exchange-format. In practice: GML (Application Schema). Most users are not able to directly digest complex GML in their applications (or don't want to spend time on that). For over 10 years we run the Open Source project NLExtract to convert these datasets into formats like PostGIS and CSV that can be directly used in applications. NLExtract itself builds on Stetl , an Open Source geospatial ETL-library in Python. NLExtract/Stetl is used a lot, but still will require users to install/maintain an NLExtract/Stetl installation, download datasets, run the ETL, check results etc. That may also be a bridge too far, if e.g. all a customer needs is a one-time CSV of say, all 10 million addresses in The Netherlands. So GeoToko taps into this niche, providing ready-to-use, often enriched Dutch datasets. Pricing is reasonable, compared to other providers, plus organizations and individuals may get highly reduced pricing when they work on Open data applications like OpenStreetMap.

As for development: the GeoToko webshop was developed with Django, Flask/nginx (Download management), Stripe (backend payments) and CKAN . The latter to provide a product-catalogue, sample data and metadata in general, all via GeoCatalogus.nl .

Further development concerned automating the NLExtract/Stetl ETL-processes. For this a framework called GeoFabriek, "GeoFactory", was developed. This automates the entire chain from checking new dataset versions at Kadaster, through downloading, conversions, packing for download and updating the metadata (in GeoCatalogus.nl).

6. Open Source Contributions

Continuous work as a contributor. Apart from some GitLab Projects, you can find/follow me best on GitHub .

GitHub contributions

New in 2021:

Ongoing in 2021:

More Contributions - Handy Docker Images

To support many of the Cloud services and Open Source projects, I develop and maintain handy Docker Images , also available from my DockerHub . Ongoing in 2021. In progress of moving all to Github Workflows build/push, replacing DockerHub Autobuild (still awaiting DockerHub Open Source waiver...):

7. Conferences - Attended

FOSS4G Buenos Aires needs special mention. As an online conference, apart from amazing content, I was very impressed with the technical and organizational setup. Even the social events were fun to join. Folks are tired of Zoom, Teams, etc but the FOSS4G LOC had introduced tools like Work Adventure and StreamYard (also used by OSgeo.nl in Big Geo Show) to enable more "cozyness". Offcourse nothing can replace a real event, but this came close. The Gala Dinner was really fun!

Jody Garnett words it very well in his Tweet:

Never forget, always in our hearts: Malena Libman (1983-2021). FOSS4G 2021 was also a conference dedicated to, and in memory of, Malena Libman. She was a pillar of the FOSS4G community and a leading organizer of this conference.

Malena was the right and truthful winner of the Sol Katz Award 2021. As the OSGeo.org website states : "Malena Libman is the Sol Katz Award winner 2021. The prize was presented at the FOSS4G closing session and handed over to her friends & family who accepted the award on her behalf since we sadly lost Malena in April 2021 to COVID-19."

8. Talks & Workshops - Provided

Below talks and workshops I provided in 2021. A complete list of presentations is available.

  • "Ten Years of OSGeo.nl - A Story in Pictures" - De Grote Geo Show - Episode 19 (video) - December 2, 2021 - Online - [Video recording on YouTube] - [Episode Info] .

  • "OGC API Features" - Geonovum - Kennisplatform APIs - Dutch API Strategy - Geo-extension WG - November 24, 2021 - Online - [PDF Slides by OGC] .

  • "FOSS4G-NL 2021 - Closing Words" - FOSS4G-NL 2021 - Oktober 20, 2021 - Enschede - [PDF Slides] .

  • "Zo gemakkelijk kun je geo-webdiensten uitrollen!" - FOSS4G-NL 2021 - Oktober 20, 2021 - Enschede - On GitOps for OSGeo Software - [PDF Slides] .

  • "GeoHealthCheck - A Quality of Service Monitor for Geospatial Web Services" - FOSS4G 2021 - September 30, 2021 - [HTML Slides] - [Abstract] - [Video recording on YouTube] .

  • "Doing Geospatial in Python" - FOSS4G 2021 - September 28, 2021 - Workshop (4h): introduction to performing common GIS/geospatial tasks using Python geospatial tools such as OWSLib, Shapely, Fiona/Rasterio, and common geospatial libraries like GDAL, PROJ, pycsw, as well as other tools from the geopython toolchain. - [HTML Startpage] - [Abstract] .

  • "Geonovum OGC API Testbed" - Open Geodag 2021 - September 14, 2021 - [PDF Slides] - [Video Recording] .

  • "Geonovum OGC API Testbed" - DiS Online: OGC en toegankelijke APIs - September 7, 2021 - [Event en PDF Slides] .

  • "Read BAG (Dutch Buildings & Addresses dataset) with Docker in 5 minutes" - 2021-04-08 - De Grote Geo Show, a live-streaming webshow by OSGeo.nl. Ep. 16 "GIS met je Toetsenbord". Aired on april 8, 2021. Video recording on YouTube .

  • "OSGeo.nl New Year's Party - Slides OSGeo.nl" - 2021-01-24 - Presentation I gave on behalf of OSGeo.nl on the online joint QGIS-NL, OpenStreetMap-NL and OSGeo.nl New Year's party on January 24, 2021.- [Slideshare]

Not as much fun as in 2019 Bukarest, but providing the Geopython Workshop at FOSS4G Buenos Aires with these great minds is still a 2021-highlight.

9. Resolutions for 2022

  • Customer portal for map5.nl
  • Get back into Kubernetes
  • Advance further into full-lifecycle GitOps
  • Advance further into Vector Tiling
  • More effort into Wegue project
  • Revive old GeoTracing projects like georambling.com
  • More favourite things
  • Whatever comes around.

January 04, 2022 01:24 PM

Nick shares problems faced by his distributed GIS team and how Mergin solved them.

Nick was searching for a way for his remote team to safely edit the GIS layers of their fibre network designs at the same time. Solutions he’d tried were either unsafe for concurrent editing, not feature-rich enough or had prohibitive licence costs. Nick now uses Mergin to collaborate with his colleagues around the world in near real time.

Highbeech logo

Nick Whittaker is Director of Highbeech Consultancy, a company providing specialised fibre optic network design services. Highbeach is based in England and operates in markets around the world.

Around 9 years ago, Highbeech started helping their clients develop city-wide fibre-to-the-home network design strategies. Nick’s team developed network simulation models which they used to understand the performance and likely construction costs of different designs fed by both public and municipality GIS data. The models proved a success as several clients commissioned construction of the resulting designs.

Nick’s team were now tasked with acting as technical liaison to the engineering companies carrying out the build in the US.

Nick Whittaker

Nick Whittaker

The Challenge

Nick now needed a way for the different project partners to safely view and edit the design at the same time: “This was working sequentially, not collaboratively - we needed to be able to work in parallel!”

“One of the engineering companies we were advising had requirements to have staff in different parts of the US and at present, everything we were doing with the designs ran on-premises. People could use VPNs but working on designs was a nightmare” Nick explained.

Nick found working on designs via VPN was too slow to be workable and instead looked for a solution which would bring designs closer to the staff needing to work on them.

Cloud storage solutions such as OneDrive, Google Drive or Dropbox are unsuitable for collaboration on GIS data as they allow files that would normally be locked and accessed one at a time to be modified simultaneously by multiple users. This situation commonly results in data loss.

“We were having to limit ourselves to one person working on the design at a time and having to notify each other when it was safe for the next person to take over - file locking by email. This was working sequentially, not collaboratively - we needed to be able to work in parallel!” he added.

Nick searched for better solutions for allowing the various teams to work together quickly and safely at the same time.

“In the past we’d used ArcGIS Server which worked well as a collaboration platform when everyone was in the same office but wouldn’t be suitable in this case. We’d also tried ArcGIS Online but found it frustrating as it was too cut-down compared with ArcGIS Server.” Nick said.

Licencing was also an important factor for Nick: “Some solutions we looked at were plugins to other suites, for example, AutoCAD. If I tell my client that we’ll do a project in AutoCAD, they may tell me they can’t afford an AutoCAD licence.”

“While searching for collaborative GIS platforms I came across the Mergin plugin for QGIS.”

QGIS is open source GIS software with a large number of extensions called plugins.

“GIS is widely used in fibre network design and I used ArcGIS primarily. I first heard about QGIS when using FiberPlanIT as it’s implemented as a QGIS plugin. Within about 12 months I was doing the majority of my GIS tasks in QGIS and now consider myself a QGIS convert, purely because I see the power, potential and capability of its community.”

Example Fibre Network Design

A fibre network design shown in QGIS

Implementation and Outcomes

After taking time to evaluate Mergin within Highbeech, Nick proposed it as a collaboration platform to the US-based engineering company.

“Initially they were unsure how to set it up so I offered to do that and to administer it for them. Within 2-3 weeks they had 20 guys using it and within 3 months I’d migrated control of it over to them.” Nick said.

“The value is all about collaborating in near real time rather than the days it used to take to do things”

Nick’s team can now see the changes their US-based client makes to the design as they make them and jump in/out as required to perform validations and make corrections and changes as required.

“The time zone difference with the US now works to our advantage - we even tell our clients we can turn their 8 hour working day into a 16 hour working day.”

“Once our client’s fielding team has finished for the day, we work through their data, performing checks and validations and everything’s done by the time they’re back in the office the next day. This forms much of what Highbeech does on the project nowadays and this wouldn’t be possible without Mergin.”

When asked what worked particularly well about Mergin, Nick said: “the seamless way you can be working with someone on a call, make a change to a design and have it appear in front of them within seconds. This is incredible and something that happens on a regular basis.”

“The value is all about collaborating in near real time rather than the days it used to take to do things. That’s the greatest strength of both Mergin and Input.”

Input is a mobile app that allows GIS projects to be viewed and edited in the field.

When asked about how collaborative working might change in the future, Nick said: “We live in a world where an ever increasing number of people choose to work from home. For those working in fibre network design, that’s only possible with collaborative platforms such as Mergin.”

Nick has since introduced other clients in the US and UK to Mergin which supports their collaborative GIS efforts together.

Download Input Today

Screenshots of the Input App for Field Data Collection

Get it on Google PlayGet it on Apple store

January 04, 2022 08:00 AM

Offset curves (also known as parallel curves) are an oft-requested feature in JTS.  They are a natural extension to the concept of buffering, and are useful for things like placing labels along rivers.  As far as I know there is no hard-and-fast definition for how an offset curve should be constructed, but a reasonable semantic would seem to be "a line lying on one side of another line at a given offset distance"

Here's an image of offset curves on both sides of a river reach from a US rivers dataset:

GEOS Implementation

GEOS acquired an implementation to produce offset curves a few years ago.  However, it has some known bugs (such as producing disconnected curves, and including extraneous segments).  It also has the feature (or quirk?) of potentially producing a result with multiple disjoint curves for a single input line. 

The GEOS implementation is based on the concept that an offset curve is just a portion of the corresponding buffer boundary.  So to generate an offset curve the algorithm extracts a portion of the buffer polygon boundary.  The trick is deciding which portion!  

The GEOS implementation generates a raw offset curve (potentially full of self-intersections and unwanted linework) and then determines the intersection of that curve with the buffer boundary.  However, the use of intersection on linework is always tricky.  The buffer geometry is necessarily only a (close) approximation, and the buffer algorithm takes advantage of this to use various heuristics to improve the quality and robustness of the generated buffer linework.  This can cause the buffer linework to diverge from the raw offset curve.  The divergence makes the intersection result susceptible to errors caused by slight differences between the generated curves. The two issues above are caused by this limitation. 

JTS Implementation

Instead of using intersection, an effective technique is to match geometry linework using a distance tolerance.  This is the approach taken in the new JTS Offset Curve algorithm. The high-level design is

  1. The buffer is generated at the offset distance
  2. The raw offset curve is generated at the same distance
  3. The raw curve segments are matched to the buffer boundary using a distance tolerance
  4. The offset curve is the section of the buffer boundary between the first and last matching points.
To make the matching efficient a spatial index is created on the buffer curve segments.  

This algorithm provides the following semantics:
  • The offset curve of a line is a single LineString
  • The offset curve lies on one side of the input line (to the left if the offset distance is positive, and to the right if negative)
  • The offset curve has the same direction as the input line
  • The distance between the input line and the offset curve equals the offset distance (up to the limits of curve quantization and numerical precision)
This algorithm does a fine job at generating offset curves for typical simple linework.  The image below shows a family of offset curves on both sides of a convoluted line.  
This resolves both of the GEOS code issues.  It also supports parameters for join style (round, bevel, and mitre), number of quadrant segments (for round joins) and mitre limit:
Join Style = MITRE
Join Style = ROUND; Quadrant Segments = 2

There are also a few nuances required to handle some tricky situations; in particular, cases when the input line string curves back on itself and when it self-intersects.  These do not have a clear definition for what the result should be.  Moreover, the algorithm itself imposes some constraints on how these cases can be handled.  The images below show how the algorithm behaves in these cases.

"Loopbacks"produce offset curves representing only the "exposed" sections of the input linework: 
Offset Curves of a Line with "loop-backs"

For a self-intersecting input line, the offset curve starts at the beginning of the input line on the specified side, and continues only up to where the line side changes due to the crossing.  The length of the offset curve is also reduced by the requirement that it be no closer than specified offset distance to the input line: 
Offset Curves of a Line with a self-intersection

This algorithm is now in JTS, and has been ported to GEOS.



by Dr JTS (noreply@blogger.com) at January 04, 2022 04:34 AM

January 02, 2022

Última atualização Dec 10, 2021.

🏠 Pessoal:

A passagem de ano foi na tranquilidade de casa com minha esposa e os Pet’s (Bella 🐶 e José 🦜).

Com o início de 2022 novas metas foram traçadas aqui em casa, dentre elas estão mais cuidados com a saúde e uma rotina de leitura com no mínimo 12 livros por ano.

Onde de COVID19 no trabalho, tivemos que reduzir a quantidade de pessoas pela metade. Dia 14 fiz um teste de COVID19 e deu negativo 🤗, contudo gripei no dia seguinte pela primeira vez em dois anos, desde o começo da pandemia, tive febre e fiquei um dia de cama 🤒. Estou melhorando, porém, minha esposa também gripou.

⚒ Profissional:

Estou estudando Expressão regular – REGEX ( curso da Alura). Eu já utilizava REGEX para algumas finalidades, mas precisava de uma base um pouco mais sólida.

❤ Comunidade:

Lançarei mais vídeos no meu canal, focando principalmente tópicos sobre o QGIS, SQL e OSM.

📚 Leitura:

Estou finalizando a leitura de O Livro do CháKakuzo Okakura e espero finalizar esse mês o História da sua vida e outros contosTed Chiang .

Meu perfil no Skoob

Nota de transparência: os links dos livros acima têm código de afiliado. Clicando neles, os preços não mudam, mas posso ganhar uma comissão da Amazon.

📺 Assistindo:

Finalizei a segunda temporada de The Boys e essa foi de explodir cabeças, literalmente. A 4ª temporada de Cobra Kai foi difícil de acompanhar, em breve teremos o Mr. Miyagi de volta.

Ainda nas séries acompanhei Hawkeye, gavião arqueiro, e a Kate Bishop é uma das melhores coisas que dessa nova etapa da Marvel. Falando de Marvel assisti Eternos e foi diferente, aguardando para ver como será o desenrolar dessa história.

Outro universo que eu e minha esposa mergulhamos foi Euphoria. Relutei um pouco para começar a assistir, pois achava que seria pesada para o momento que vivemos (PS: É pesada sim).

🎧 Ouvindo:

Em janeiro não pode faltar David Bowie 👨‍🎤 no meu repertório, e indicarei o álbum Hunky Dory .

Frase do momento


“O tempo pode me mudar, e eu não posso enganar o tempo”

— David Bowie

The post Janeiro / 2022 appeared first on Narcélio de Sá.

by Narcélio de Sá at January 02, 2022 11:30 PM

January 01, 2022

I have not done the year-end blog post very often, but it’s cold and snowy outside, I have my toes under a knit blanket, and like Oprah I feel like sharing some of my favourite things.

Oprah

No, not sweater capes, calypso music, paisley tops, or Chinese checkers but some things that made last year bearable to me that other folks might also enjoy.

Getting Strong

I spent a lot of time in the gym as a young man, and for a while I had the shoulders to prove it, but as is the nature of things over time I invested less time in my physique (and more in family and work) so the muscles mostly melted away.

I always missed the post-gym feeling, but while I accumulated the start of a home gym over the years, I never gave it a lot of use, until Covid came. Working from home, the garage gym is only ever a flight of stairs away, so putting in 3 or 4 hours a week is not too hard.

Weights

My overarching goal is to exercise 7 hours a week, so I do 3 or 4 hours of strength training, and fill in the remaining hours with yoga, rowing, walking or (until an injury this spring) running.

In terms of positive returns, this has been one of the best investments I have made in the past 10 years. I sleep better, my formerly sore back is no more, and I have a body that would make my 35-year-old self green with jealousy.

It also turns out I’m in the vanguard of the conventional wisdom! The New York Times seems to publish a new study on the superior health benefits of strength training every day.

There is a huge amount of information online about getting started, and I have only one non-conventional piece of advice for new folks: no matter what your age, look at the “weight training over 50” articles. The routines for older people tend to stress “full chain” movements (squats, pull-ups, push-ups) and slower progressions to avoid injury. The result is a stronger core and back which are a key foundation to build general fitness. (And supplemental, be sure to read Zeynep Tufekci’s amazing article on the lies the exercise industry tells women and how everyone can get strong.)

Finally (this has gotten long) doing yoga regularly has allowed me to strengthen all kinds of tiny muscles I never knew I had, but that turn out to be critical to core strength useful in other strength training moves. Since Covid shut my local yoga studio I have been using the Down Dog app on my phone to guide my sessions, which has been money well spent.

Revolutions Podcast

There’s probably an entry for this in the Stuff White People Like (the Stuff Older White Guys Like?) but c’est la vie: Mike Duncan’s Revolutions Podcast has been a constant companion both in the garage gym and while out walking Victoria’s lovely coastline.

Revolutions

Duncan is currently wrapping up his tenth revolution (the Russian revolutions) and this series has been the work of multiple years so there are hundreds of episodes of content.

  • English Revolution (16 episodes)
  • American Revolution (15 episodes)
  • French Revolution (55 episodes)
  • Haitian Revolution (19 episodes)
  • Spanish American wars of independence (27 episodes)
  • July Revolution (7 episodes)
  • Revolutions of 1848 (33 episodes)
  • Paris Commune (8 episodes)
  • Mexican Revolution (27 episodes)
  • Russian Revolution (70+ episodes)

I found that for historical periods I knew something about, like the French Revolution, the podcast filled in the 90% that I was missing, and it was enthralling stuff. The Revolutionary wars in Europe, the ebb and flow of the power of the Parisian street, the whole period of the Directory! Turns out that just reading Tale of Two Cities doesn’t give you enough backgrounding in the Revolution.

Revolutions would be worth a listen just for the extra context and detail on things I already knew, but the real eye openers have been the revolutions I knew basically nothing about.

OK, I knew there was a Haitian revolution. A slave uprising, right? That’s the sum total I went in with. Revolutions has 19 episodes on the Haitian revolution, and the whole thing is amazing. Tragic, uplifting, depressing, hair-tearing, absolutely worth knowing about.

Similarly the Mexican revolution. Sure, there had to have been one, right? Pancho Villa, he’s a guy who did some stuff? I’m in the midst of it now (Revolution number 9) and the historical echoes into the present day are eerie.

Anyways, the Revolutions Podcast, give it a try. Middle aged white guys and everyone else.

Ursula Le Guin

Not for the first time, but returning to Le Guin this year has been very comforting. Her prose is so clear and unfussy and easy to read, yet also contains so much beauty and observation. In particular I have enjoyed revisiting:

  • The Dispossessed, which structurally owes a lot to Siddhartha or The Glass Bead Game. The life story of a philosopher finding his way to understanding. But this philosopher lives in a world of perfect equality, a world-wide anarcho-syndicalist society, and as in all (good) sci-fi the fun is in unravelling the ways humans live in this totally foreign millieu.
  • The Found and the Lost, a huge collection of novellas, ranging from sci-fi pieces set in the Hainish universe to fantasy pieces in Earthsea. They are all excellent, but the one that sticks with me the most is Matter of Segri. A world in which there are 16 women for every man, and the men are kept separate in “castles” from the age of 12 on, “castles” within which they form elabourate dominance heirarchies, torment each other, and practice displays of strength. So, like, a total fantasy cough.

Art Tatum

I am a half-assed jazz pianist, and my interest in the details of the art form comes and goes, but at some point I heard someone say that, if you want to understand jazz piano you have to listen to Art Tatum. In particular, listen to the embellishments.

Art Tatum

So I did.

The first thing that hits you is the technical perfection of his playing. Every scale, every arpeggio, the incredible regulation of his touch. I mean, listening to professional piano players, of course the technique is amazing, but Tatum is a notch above.

The second thing, particularly in the early work, where he’s just banging out popular tunes of the time, is the ad-libbed fills and transitions, the embellishments. The harmonic structure he uses often feels appropriate to recordings from 10 or 15 years later on. I totally get where the commentator I read was coming from: Tatum is a precursor to later players.

Check out this jaunty stride rendition of Tea for Two. Crazy fills and asides, and getting increasingly harmonically fun as the piece goes on. Three minutes of happiness.

Anyways, make some time in your streaming playlist for Art Tatum.

January 01, 2022 08:00 AM

For Window users, several new GRASS GIS addons that were published recently did not show up in the list of available addons when running the g.extension function. One of these was the r.suitability.regions addon I wrote some time back back (check out the accompanying tutorial here).

But as a great start of the new year, this has now been fixed. A big thanks to the dev team! So, did you see an interesting addons mentioned on e.g., twitter, but were not able yet to install it yet? Time to try it again :-).

January 01, 2022 12:00 AM

December 31, 2021

Thanks to Louis Maniquet https://unsplash.com/@louis_mna

Another year passes with a record low of blog posts. My excuse/explanation is that Ed Freyfogle and I have put a lot of time into the Geomob Podcast – over 50 conversations this year including OpenStreetMap, Earth Observation, the Locus Charter, cartography and map based art, marketing, open source, interviewing several book authors, drones, neogeography (remember that?), hobby projects, products and politics – phew that really is a lot isn’t it?

In December 2020 we recorded a Christmas Special with a several of our interviewees, as part of that episode they made predictions for what might happen in 2021, purely for fun as none of us expected the predictions to be accurate. You can read the list of 2021 predictions and then listen to this year’s Christmas Roast episode and the follow up Christmas Pudding episode to hear how our friends wriggled out of their mis-predictions and to hear their thoughts for the coming year.

To make it easier for me to ridicule their pitiful efforts at crystal ballery, here is a summary of their predictions for 2022.

  • Denise McKenzie: more data regulation, sharper focus on data governance
  • Ed Parsons: recognition that we don’t have the data we need for the metaverse
  • Mark Iliffe:
    – more commoditisation of geospatial (cites the example of Excel)
    – whatever Ed P says will be wrong
  • Steven Feldman:
    – An in-person Geomob in a new city!
    – Geounicorn event (a geo business going public or selling for over $1bn
    – all the “action” will be in remote sensing. Look to the skies!
  • Alex Wrottesley:
    Much more effort from organizations trying to find tools and datasets to
    try to make sense of environmental guidance and targets around climate change.
    Specifically energy performance, verification of compliance with targets,
    the natural environment, etc. Big need for better understanding and tools in
    this space.
  • Ed Freyfogle: More innovation around delivery services, courier services
  • Ken Field:
    – decline of data dashboards
    – a new term for “data scientists”
    – increase in the use of Natural Language Processing
  • Jeremy Morley: more focus on the robustness of GNSS and location system timing

I have one further prediction to make – when we get together in December 2022 all of my mates (and I) will be saying something along the lines of “well it sort of happened, it’s just taking a bit longer than a year”

Happy new year everyone

by Steven at December 31, 2021 01:28 PM

What a ride.

This year has been both extremely rewarding and incredibly frustrating, sometimes both in very short succession.

I’ve finally finished my PhD dissertation and – between movement data analysis and open source and open data science talks in general – I’ve been counting over ten invited talks and conference presentations, including keynotes at FOSS4G and GI_Forum. Unfortunately, all of these were limited to virtual experiences and therefore often lacked much of the social interaction off stage that usually makes giving talks rewarding. But FOSS4G2021 was a refreshing exception to this rule:

One of the rare in-person events I attended this year was the Futurezone 2020 Award ceremony which – of course – had been postponed to 2021. The award took me completely by surprise:

This year’s focus on talks meant that there haven’t been many blog posts with original content this year, an unfortunate situation I hope to improve in 2022.

Without wanting to promise to much, there are quite a few interesting MovingPandas collaborations in the works that will hopefully result in exciting new features, demos, and tutorials:

On the plus side, with so many virtual events – from conferences to community events such as QGIS Open Days – much of the content formerly exclusively available to participants on-site have been recorded. Some worthwhile accounts and playlists include:

Happy streaming, happy new year, and – if you can – get vaccinated!

I’ll be having a nerd party.

by underdark at December 31, 2021 12:19 AM

There are many reasons to like the GeoPackage. One is that it makes it so easy to share your data, including symbology and other layer properties in one database file. And what makes this even better is that you can also store QGIS projects directly inside GeoPackage files. This gets you a totally self-contained GeoPackage that you can easily share.

Saving layer styles to a Geopackage

One way to include layer properties is to save them for individual layer to the GeoPackage. To do so, go to the Symbology tab in the Layer properties windows. Select 1 Style > save style and in the popup window, choose the option 2 In Database (Geopackage).

Optionally, select the option 3 Use as default style for this layer. This is especially convenient if you want to save more than one style for a layer.

One disadvantage is that it does not seem possible yet to save the symbology and other properties of raster layers. In addition, the layers metadata is not included. This is because the metadata is normally saved to the project file.

Saving metadata to a Geopackage

If you want to store the metadata of a layer together with the layer in a Geopackage, go to export > save as option in the layers context menu and make sure to select the option Persist layer metadata.

You can also use the Package layers function in the Processing toolbox to save multiple layers at once to a (new) GeoPackage. Here, you need to select the option Save layer metadata into Geopackage option.

For now it only seems possible to save the metadata of vector layers. That means that if you want to share the metadata of a raster layer, you need to save it as a QMD file. Or, see below how you can save a whole project, including the metadata of raster layers, in a GeoPackage.

Store a project to a Geopackage

Another option is to store QGIS projects directly inside a GeoPackage file. This way, you can create a completely self-contained GeoPackage with not only the data and accompanying styling, but also the project information and settings! Note that because the layer metadata is stored in the project file, this way you can also save the metadata of raster layers in your GeoPackage.

If you already have a project, all you have to do is to go open the Project menu and select Save to > GeoPackage option.

Now, to open the project later on, go to the Project menu and select the option Open From > Geopackage.

Note that you can store multiple projects in a GeoPackage. So you create different projects with for example different styles or with project settings that serve different purposes or audiences.

Also noteworthy, you can save projects to your PostgreSQL database as well. Either way, the ability to store data with accompanying metadata and styling in one place makes it a whole lot easier to store, handle and sharing data.

December 31, 2021 12:00 AM

December 30, 2021

So 2021 wasn’t much better than 2020. Another year of endless virtual meetings and the 24 hour office. Here are some updates from WFH life: pygeoapi: both OGC API – Records and OGC API – Environmental Data Retrieval support were added to the codebase. The project also saw both CQL and i18n support, which is […]

by tomkralidis at December 30, 2021 02:15 PM

December 29, 2021

Today’s post is a follow-up and summary of my mapping efforts this December. It all started with a proof of concept that it is possible to create a nice looking snowfall effect using only labeling:

After a few more iterations, I even included the snowflake style in the first ever QGIS Map Design DLC: a free extra map recipe that shows how to create a map series of Antarctic expeditions. For more details (including project download links), check out my guest post on the Locate Press blog:

If you want to just use the snowflake style in your own projects, the easiest way is to grab the “Snowy Day” project from the QGIS hub (while the GeoPackage is waiting for approval on the official site, you can get it from my Dropbox):

The project is self-contained within the downloaded GeoPackage. One of the most convenient ways to open projects from GeoPackages is through the browser panel:

From here, you can copy-paste the layer style to any other polygon layer.

To change the snowflake color, go to the project properties and edit the “flake_color” variable.

Happy new year!

by underdark at December 29, 2021 11:35 AM

December 27, 2021

Early in 2020 I had the idea of registering for races outside Colorado in 2021. You know, after the pandemic would be over. One of the classic Mid-western ultra-marathons, if I won the lottery. Sadly, these races were cancelled in 2020 and no clear end to the pandemic was in sight at the end of the year. I decided to stick to races that wouldn't require flights and accomodations. I would run the relatively small races in my neighborhood, all of which I've run before, but in 2021 I would run all of them in 6 months and complete the Gnar Slam.

Here are the numbers for 2021 (as of 2021-12-27).

  • 410 hours, 52 minutes of running (18% more than in 2020).

  • 2001 miles distance (+15%)

  • 264,514 ft of elevation gain (+17%)

  • 6 pairs of shoes retired

https://live.staticflickr.com/65535/51059987346_5bcbfccebb_c.jpg

March's big snowstorm complicated training

My achievements for the year include:

  • Completing the Gnar Slam (Quad Rock 50 mile, Never Summer 100 km, Black Squirrel Trail Half-Marathon, Blue Sky Trail Marathon).

  • 3rd place Men's Grandmasters at Quad Rock, my first racing trophy.

  • New personal bests at Quad Rock, Never Summer, and Blue Sky.

  • Finishing my own backyard ultra-marathon, 7 x 4.16 miles in March.

The low points of my 2021 season were being injured for most of March and half of April, and mental fatigue from the corporate union-busting at my day job (gaslighting, vague threats, pervasive fear, uncertainty, and doubt are part of the playbook) which undercut my Black Squirrel training. Happily, I fully recovered from the injury just before Quad Rock, and found myself a new job just before Blue Sky.

Many things went right in 2021. I did a bunch of long alpine runs with Mike Thompson and look forward to doing more of that in 2022. I got a lot better at descending trails. I'm faster, more confident, and use less energy going downhill. I was only injured once and it didn't require a trip to the emergency room. I wasn't attacked by a mountain lion. All in all, a very positive year. I'm looking forward to the next one. I don't have a firm plan for 2022 yet, but will probably run a little less and not do any ultra-marathon event until the end of summer.

https://live.staticflickr.com/65535/51072492107_4cc8c166aa_c.jpg

Fresh mountain lion tracks on Horsetooth's South Ridge, March 24, 2021

I've gathered photos that appear in previous posts into a Flickr album.

https://live.staticflickr.com/65535/51123795163_00eeafb1a8_b.jpg

Milner Mountain, April 17, 2021

by Sean Gillies at December 27, 2021 03:59 AM

December 24, 2021

Recording of the papers presented at the 17th International gvSIG Conference, which were held online from December 1 to 3, are now available. Besides, the presentations in ODP and/or PDF format are included.

Recording of the workshops on “Mapping impact and extension of fires with gvSIG” and “Advanced editing and version control system in gvSIG Desktop“, both in Spanish, are also available with the information with the cartography to be used.

If you couldn’t follow the live conference you can watch the videos of the different presentations and the workshops, and download the presentations now.

Don’t miss them and see you in the next gvSIG Conference!

by Mario at December 24, 2021 11:23 AM

Ya están disponibles las grabaciones de las ponencias presentadas en las 17as Jornadas Internacionales gvSIG, que se celebraron de forma online del 1 al 3 de diciembre. Aparte se incluyen las presentaciones en formato ODP y/o PDF.

También está disponible la grabación de los talleres sobre “Mapeo del impacto y extensión de incendios con gvSIG” y “Edición avanzada y control de versiones en gvSIG Desktop” junto a la información con la cartografía a utilizar.

Si no pudiste seguir las jornadas en directo ahora puedes disponer de la grabación de las distintas ponencias, y también de los talleres realizados.

¡Os esperamos en las siguientes jornadas!

by Mario at December 24, 2021 11:08 AM

December 23, 2021

December 22, 2021

The GeoTools team is pleased to announce the release GeoTools 25:    geotools-25.4-bin.zip    geotools-25.4-doc.zip    geotools-25.4-userguide.zip    geotools-25.4-project.zipGeoTools 25.4 is published to repo.osgeo.org for direct build integration. This release is made in conjunction with GeoServer 2.19.4. The GeoTools team is grateful to GeoSolutions for allowing Andrea Aime the time to

by Andrea Aime (noreply@blogger.com) at December 22, 2021 05:22 PM

The GeoServer team are happy to announce GeoServer 2.19.4 release is available for download (zip and war) along with docs and extensions.

This GeoServer 2.19.4 release was produced in conjunction with GeoTools 25.4 and GeoWebCache 1.19.2, this is a maintenance release recommended for production systems.

Thanks to everyone who contributed, and to Andrea Aime (GeoSolutions) for making this release.

Improvements and Fixes

Bug

  • GEOS-10337 Harden importer against failed imports, make failures more evident

  • GEOS-10322 JDBCConfig community module does not deal with stale connections to the database

  • GEOS-10300 The map preview logs errors when using AUTO codes

  • GEOS-10299 The reprojection console does not work with AUTO codes

  • GEOS-10292 Changing worker pool size in raster access is not actually applied (silent error)

  • GEOS-10289 GeoServer busy for 1 hour on reloading a 50000 shapefiles Directory datastore

  • GEOS-10281 GeoServer log level not picked up with Catalog reload

  • GEOS-10249 GWC produce NPE when it comes to race condition

Improvement

  • GEOS-10328 Expire completed and stale importer contexts

  • GEOS-10321 WCS 2.0 might fail to return coverages whose native BBOX goes slighly outside of the dateline

  • GEOS-10315 Features Templating - Allow injecting JSON-LD output in HTML

  • GEOS-10314 Features Templating - allow specifying root @type in the JSON-LD output and a different name for features array

GEOS-9904 GeoFence backend DBMS dependencies

Task

  • GEOS-10335 Update GeoServer to a log4j version that does not support RCEs

  • GEOS-10269 Overriding JSON Object while Merging Feature Templates

  • GEOS-10268 Null Support in Features Templating

About GeoServer 2.19

Additional information on GeoServer 2.19 series:

WMS GetFeatureInfo includes labels from ColorMap

Promote WMTS multidim to extension

Promote WPS-Download to extension

Promote params-extractor to extension

Promote GWC-S3 to extension

Promote WPS-JDBC to extension status

Promote MapML to extension status

GeoServer repository transition to main branch

by GeoServer community at December 22, 2021 12:00 AM