howto_generate_database.md 4.46 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Download pbf file
=================

I sure you can do that yourself


Build elevation database
========================

This database is not usefull for routing. This database is only usefull for
building routing database. This database is not needed to be generated when the
pbf is updated if the pdf cover the same area.

Analyze pbf
-----------

    ./src/analyze_pbf SOMEWHERE/france-latest.osm.pbf > SOMEWHERE/list_needed_srtm

The `analyze_pbf` program give:

 - in stdout the list of needed srtm tiles.

 - is stderr the number of couple (way,node)s usefull for routing. Let N the
Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
24
   number of couple (way,node)s returned by `analyze_pbf`. The prediction are the
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53
   following:

   - (150 Bytes) × N : the routing database size

   - (157 Bytes) × N : the minimal amount of RAM to create the database

   - (253 Bytes) × N : a /good/ amount of RAM to create the database

   - (421 Bytes) × N : the optimal amount of RAM to create the database (more
           RAM is useless).

   Keep these value in mind for next part.

Downloading SRTM tiles
----------------------

With the `list_needed_srtm`, download the tiles. A script is provided in scripts.
This script is the definition of brutality.


    ./scripts/dlsrtm SOMEWHERE/list_needed_srtm SOMEWHERE/srtmzip


Unzip SRTM files
----------------

    mkdir -p SOMEWHERE/srtm
    find SOMEWHERE/srtmzip/ -print0 | xargs -0 -n1 -I{} unzip {} -d SOMEWHERE/srtm

Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
54
We don't need `SOMEWHERE/srtmzip` anymore. You can remove it.
55 56 57 58 59 60 61 62 63 64 65 66 67


Creating elevation db
---------------------

    ./src/elevation_create_database SOMEWHERE/ele_db_sparse SOMEWHERE/srtm/*.hgt

This elevation db is directly usable. Buto create this elevation db, the program
use a writable map, and the elevation database is a big sparse file. If you are
not planning to backup, or transfer this file, you can skip the next step, but
the time used in the next step is very small, and this is not a good idea in
general case.

Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
68
We don't need `SOMEWHERE/srtm anymore`. You can remove it.
69 70 71 72

Converting elevation db
-----------------------

Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
73
We use the agnostic `lmdb_rewriter` programm. It read source database, and
74 75 76 77 78 79
re-write it sequencialy (without using writable map). The two database are
equivalent, but the created elevation db from last step is a big sparse file,
and after this step, the elevation db is a small dense file.

    ./src/lmdb_rewriter SOMEWHERE/ele_db_sparse SOMEWHERE/ele_db
    
Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
80
`SOMEWHERE/ele_db_sparse` is not needed anymore.
81 82 83 84 85 86 87 88


Keep it
-------

Keep this elevation database. It is not needed to recreate it when the pbf is
updated. Except the case when new area is added to the pbf which are not in the
elevation database, but in this case, you will know it, the routing db (next
Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
89
part) creation will fail, with a explanation (maybe).
90 91 92 93 94


Testing the elevation database
------------------------------

Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
95
The program `elevation_query_database` use coupe (lon,lat) in stdin and give
96 97 98 99 100 101 102 103 104 105 106 107
elevation in stdout. For example, to get the elevation of the point given by
coords (lon=1.017, lat=48.513) use:

    echo 1.017 48.513 | ./src/elevation_query_database SOMEWHERE/ele_db


Building routing database
=========================

You need to rebuild it each time the pbf is updated (if you need a updated
routing, of course). To create this database, you need to have:

Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
108
 - the pbf (obviously)
109

Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
110
 - a elevation database which cover the area
111

Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
112
 - a large amount of RAM (see predictions in previous part obtained by pbf
113 114
  analysis)

Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
115

116 117 118 119 120 121 122 123 124 125 126
    ./src/route_create_databases_from_pbf SOMEWHERE/france-latest.osm.pbf SOMEWHERE/ele_db SOMEWHERE/route_db

Have a cup of tea.

Testing routing database
------------------------

The routing database is made by two databases, the lookup database, and the node
database.

First we test the lookup database. For that the program
Jean-Benoist Leger's avatar
Jean-Benoist Leger committed
127
`route_query_lookup_database` do the job. This program take a set of coupe
128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146
(lon,lat) terminated by 0. For example, to get the nodes in the same connected
component which are near points of coords:
 - (lon=0.044, lat=45.661)
 - (lon=0.169, lat=45.648)
 - (lon=0.240, lat=45.737)

use the following command:

    echo 0.044 45.661 0.169 45.648 0.240 45.737 0 \
      | ./src/route_query_lookup_database SOMEWHERE/route_db

Second we test the node db. You must have node id (you can use nodes id found by
the lookup, with this you can also verify node are near queried coordinates).
For example, to obtain information about the nodes 1068822559 and 561188093, use
the following command:

    echo 1068822559 561188093 \
      | ./src/route_query_nodes_database SOMEWHERE/route_db