1r.in.xyz(1) GRASS GIS User's Manual r.in.xyz(1)
2
6 r.in.xyz - Creates a raster map from an assemblage of many coordinates
7 using univariate statistics.
8
10 raster, import, statistics, conversion, aggregation, binning, ASCII,
11 LIDAR
12
14 r.in.xyz
15 r.in.xyz --help
16 r.in.xyz [-sgi] input=name output=name [method=string] [separa‐
17 tor=character] [x=integer] [y=integer] [z=integer] [skip=inte‐
18 ger] [zrange=min,max] [zscale=float] [value_column=integer]
19 [vrange=min,max] [vscale=float] [type=string] [percent=integer]
20 [pth=integer] [trim=float] [--overwrite] [--help] [--verbose]
21 [--quiet] [--ui]
22 Flags:
24 -s
25 Scan data file for extent then exit
26 -g
28 In scan mode, print using shell script style
29 -i
31 Ignore broken lines
32 --overwrite
34 Allow output files to overwrite existing files
35 --help
37 Print usage summary
38 --verbose
40 Verbose module output
41 --quiet
43 Quiet module output
44 --ui
46 Force launching GUI dialog
47 Parameters:
49 input=name [required]
50 ASCII file containing input data (or "-" to read from stdin)
51 output=name [required]
53 Name for output raster map
54 method=string
56 Statistic to use for raster values
57 Options: n, min, max, range, sum, mean, stddev, variance,
58 coeff_var, median, percentile, skewness, trimmean
59 Default: mean
60 n: Number of points in cell
61 min: Minimum value of point values in cell
62 max: Maximum value of point values in cell
63 range: Range of point values in cell
64 sum: Sum of point values in cell
65 mean: Mean (average) value of point values in cell
66 stddev: Standard deviation of point values in cell
67 variance: Variance of point values in cell
68 coeff_var: Coefficient of variance of point values in cell
69 median: Median value of point values in cell
70 percentile: Pth (nth) percentile of point values in cell
71 skewness: Skewness of point values in cell
72 trimmean: Trimmed mean of point values in cell
73 separator=character
75 Field separator
76 Special characters: pipe, comma, space, tab, newline
77 Default: pipe
78 x=integer
80 Column number of x coordinates in input file (first column is 1)
81 Default: 1
82 y=integer
84 Column number of y coordinates in input file
85 Default: 2
86 z=integer
88 Column number of data values in input file
89 If a separate value column is given, this option refers to the
90 z-coordinate column to be filtered by the zrange option
91 Default: 3
92 skip=integer
94 Number of header lines to skip at top of input file
95 Default: 0
96 zrange=min,max
98 Filter range for z data (min,max)
99 zscale=float
101 Scale to apply to z data
102 Default: 1.0
103 value_column=integer
105 Alternate column number of data values in input file
106 If not given (or set to 0) the z-column data is used
107 Default: 0
108 vrange=min,max
110 Filter range for alternate value column data (min,max)
111 vscale=float
113 Scale to apply to alternate value column data
114 Default: 1.0
115 type=string
117 Type of raster map to be created
118 Storage type for resultant raster map
119 Options: CELL, FCELL, DCELL
120 Default: FCELL
121 CELL: Integer
122 FCELL: Single precision floating point
123 DCELL: Double precision floating point
124 percent=integer
126 Percent of map to keep in memory
127 Options: 1-100
128 Default: 100
129 pth=integer
131 Pth percentile of the values
132 Options: 1-100
133 trim=float
135 Discard <trim> percent of the smallest and <trim> percent of the
136 largest observations
137 Options: 0-50
138
140 The r.in.xyz module will load and bin ungridded x,y,z ASCII data into a
141 new raster map. The user may choose from a variety of statistical meth‐
142 ods in creating the new raster. Gridded data provided as a stream of
143 x,y,z points may also be imported.
144 Please note that the current region extents and resolution are used for
146 the import. It is therefore recommended to first use the -s flag to get
147 the extents of the input points to be imported, then adjust the current
148 region accordingly, and only then proceed with the actual import.
149 r.in.xyz is designed for processing massive point cloud datasets, for
151 example raw LIDAR or sidescan sonar swath data. It has been tested with
152 datasets as large as tens of billion of points (705GB in a single
153 file).
154 Available statistics for populating the raster are (method):
156 n number of points in cell
158 min minimum value of points in cell
160 max maximum value of points in cell
162 range range of points in cell
164 sum sum of points in cell
166 mean average value of points in cell
168 stddev standard deviation of points in cell
170 variance variance of points in cell
172 coeff_var coefficient of variance of points in cell
174 median median value of points in cell
176 percentile pth percentile of points in cell
178 skewness skewness of points in cell
180 trimmean trimmed mean of points in cell
182 · Variance and derivatives use the biased estimator (n). [subject
185 to change]
186 · Coefficient of variance is given in percentage and defined as
188 (stddev/mean)*100.
189 It is also possible to bin and store another data column (e.g.
191 backscatter) while simultaneously filtering and scaling both the data
192 column values and the z range.
193
195 Gridded data
196 If data is known to be on a regular grid r.in.xyz can reconstruct the
197 map perfectly as long as some care is taken to set up the region cor‐
198 rectly and that the data’s native map projection is used. A typical
199 method would involve determining the grid resolution either by examin‐
200 ing the data’s associated documentation or by studying the text file.
201 Next scan the data with r.in.xyz’s -s (or -g) flag to find the input
202 data’s bounds. GRASS uses the cell-center raster convention where data
203 points fall within the center of a cell, as opposed to the grid-node
204 convention. Therefore you will need to grow the region out by half a
205 cell in all directions beyond what the scan found in the file. After
206 the region bounds and resolution are set correctly with g.region, run
207 r.in.xyz using the n method and verify that n=1 at all places. r.uni‐
208 var can help. Once you are confident that the region exactly matches
209 the data proceed to run r.in.xyz using one of the mean, min, max, or
210 median methods. With n=1 throughout, the result should be identical
211 regardless of which of those methods are used.
212 Memory use
214 While the input file can be arbitrarily large, r.in.xyz will use a
215 large amount of system memory for large raster regions (10000x10000).
216 If the module refuses to start complaining that there isn’t enough mem‐
217 ory, use the percent parameter to run the module in several passes. In
218 addition using a less precise map format (CELL [integer] or FCELL
219 [floating point]) will use less memory than a DCELL [double precision
220 floating point] output map. Methods such as n, min, max, sum will also
221 use less memory, while stddev, variance, and coeff_var will use more.
222 The aggregate functions median, percentile, skewness and trimmed mean
223 will use even more memory and may not be appropriate for use with arbi‐
224 trarily large input files.
225 The default map type=FCELL is intended as compromise between preserving
227 data precision and limiting system resource consumption. If reading
228 data from a stdin stream, the program can only run using a single pass.
229 Setting region bounds and resolution
231 You can use the -s scan flag to find the extent of the input data (and
232 thus point density) before performing the full import. Use g.region to
233 adjust the region bounds to match. The -g shell style flag prints the
234 extent suitable as parameters for g.region. A suitable resolution can
235 be found by dividing the number of input points by the area covered.
236 e.g.
237 wc -l inputfile.txt
238 g.region -p
239 # points_per_cell = n_points / (rows * cols)
240 g.region -e
241 # UTM location:
242 # points_per_sq_m = n_points / (ns_extent * ew_extent)
243 # Lat/Lon location:
244 # points_per_sq_m = n_points / (ns_extent * ew_extent*cos(lat) * (1852*60)^2)
245 If you only intend to interpolate the data with r.to.vect and
247 v.surf.rst, then there is little point to setting the region resolution
248 so fine that you only catch one data point per cell -- you might as
249 well use "v.in.ascii -zbt" directly.
250 Filtering
252 Points falling outside the current region will be skipped. This
253 includes points falling exactly on the southern region bound. (to cap‐
254 ture those adjust the region with "g.region s=s-0.000001"; see
255 g.region)
256 Blank lines and comment lines starting with the hash symbol (#) will be
258 skipped.
259 The zrange parameter may be used for filtering the input data by verti‐
261 cal extent. Example uses might include preparing multiple raster sec‐
262 tions to be combined into a 3D raster array with r.to.rast3, or for
263 filtering outliers on relatively flat terrain.
264 In varied terrain the user may find that min maps make for a good noise
266 filter as most LIDAR noise is from premature hits. The min map may also
267 be useful to find the underlying topography in a forested or urban
268 environment if the cells are over sampled.
269 The user can use a combination of r.in.xyz output maps to create custom
271 filters. e.g. use r.mapcalc to create a mean-(2*stddev) map. [In this
272 example the user may want to include a lower bound filter in r.mapcalc
273 to remove highly variable points (small n) or run r.neighbors to smooth
274 the stddev map before further use.]
275 Alternate value column
277 The value_column parameter can be used in specialized cases when you
278 want to filter by z-range but bin and store another column’s data. For
279 example if you wanted to look at backscatter values between 1000 and
280 1500 meters elevation. This is particularly useful when using r.in.xyz
281 to prepare depth slices for a 3D raster â the zrange option defines
282 the depth slice but the data values stored in the voxels describe an
283 additional dimension. As with the z column, a filtering range and scal‐
284 ing factor may be applied.
285 Reprojection
287 If the raster map is to be reprojected, it may be more appropriate to
288 reproject the input points with m.proj or cs2cs before running
289 r.in.xyz.
290 Interpolation into a DEM
292 The vector engine’s topographic abilities introduce a finite memory
293 overhead per vector point which will typically limit a vector map to
294 approximately 3 million points (~ 1750^2 cells). If you want more, use
295 the r.to.vect -b flag to skip building topology. Without topology, how‐
296 ever, all you’ll be able to do with the vector map is display with
297 d.vect and interpolate with v.surf.rst. Run r.univar on your raster
298 map to check the number of non-NULL cells and adjust bounds and/or res‐
299 olution as needed before proceeding.
300 Typical commands to create a DEM using a regularized spline fit:
302 r.univar lidar_min
303 r.to.vect -z type=point in=lidar_min out=lidar_min_pt
304 v.surf.rst in=lidar_min_pt elev=lidar_min.rst
305 Import of x,y,string data
307 r.in.xyz is expecting numeric values as z column. In order to perform a
308 occurrence count operation even on x,y data with non-numeric
309 attribute(s), the data can be imported using either the x or y coordi‐
310 nate as a fake z column for method=n (count number of points per grid
311 cell), the z values are ignored anyway.
312
314 Import of x,y,z ASCII into DEM
315 Sometimes elevation data are delivered as x,y,z ASCII files instead of
316 a raster matrix. The import procedure consists of a few steps: calcula‐
317 tion of the map extent, setting of the computational region accordingly
318 with an additional extension into all directions by half a raster cell
319 in order to register the elevation points at raster cell centers.
320 Note: if the z column is separated by several spaces from the coordi‐
322 nate columns, it may be sufficient to adapt the z position value.
323 # Important: observe the raster spacing from the ASCII file:
324 # ASCII file format (example):
325 # 630007.5 228492.5 141.99614
326 # 630022.5 228492.5 141.37904
327 # 630037.5 228492.5 142.29822
328 # 630052.5 228492.5 143.97987
329 # ...
330 # In this example the distance is 15m in x and y direction.
331 # detect extent, print result as g.region parameters
332 r.in.xyz input=elevation.xyz separator=space -s -g
333 # ... n=228492.5 s=215007.5 e=644992.5 w=630007.5 b=55.578793 t=156.32986
334 # set computational region, along with the actual raster resolution
335 # as defined by the point spacing in the ASCII file:
336 g.region n=228492.5 s=215007.5 e=644992.5 w=630007.5 res=15 -p
337 # now enlarge computational region by half a raster cell (here 7.5m) to
338 # store the points as cell centers:
339 g.region n=n+7.5 s=s-7.5 w=w-7.5 e=e+7.5 -p
340 # import XYZ ASCII file, with z values as raster cell values
341 r.in.xyz input=elevation.xyz separator=space method=mean output=myelev
342 # univariate statistics for verification of raster values
343 r.univar myelev
344 Import of LiDAR data and DEM creation
346 Import the Jockey’s Ridge, NC, LIDAR dataset (compressed file
347 "lidaratm2.txt.gz"), and process it into a clean DEM:
348 # scan and set region bounds
349 r.in.xyz -s -g separator="," in=lidaratm2.txt
350 g.region n=35.969493 s=35.949693 e=-75.620999 w=-75.639999
351 g.region res=0:00:00.075 -a
352 # create "n" map containing count of points per cell for checking density
353 r.in.xyz in=lidaratm2.txt out=lidar_n separator="," method=n zrange=-2,50
354 # check point density [rho = n_sum / (rows*cols)]
355 r.univar lidar_n
356 # create "min" map (elevation filtered for premature hits)
357 r.in.xyz in=lidaratm2.txt out=lidar_min separator="," method=min zrange=-2,50
358 # set computational region to area of interest
359 g.region n=35:57:56.25N s=35:57:13.575N w=75:38:23.7W e=75:37:15.675W
360 # check number of non-null cells (try and keep under a few million)
361 r.univar lidar_min
362 # convert to points
363 r.to.vect -z type=point in=lidar_min out=lidar_min_pt
364 # interpolate using a regularized spline fit
365 v.surf.rst in=lidar_min_pt elev=lidar_min.rst
366 # set color scale to something interesting
367 r.colors lidar_min.rst rule=bcyr -n -e
368 # prepare a 1:1:1 scaled version for NVIZ visualization (for lat/lon input)
369 r.mapcalc "lidar_min.rst_scaled = lidar_min.rst / (1852*60)"
370 r.colors lidar_min.rst_scaled rule=bcyr -n -e
371
373 · Support for multiple map output from a single run.
374 method=string[,string,...] output=name[,name,...]
375 This can be easily handled by a wrapper script, with the added
376 benefit of it being very simple to parallelize that way.
377
379 · "nan" can leak into coeff_var maps.
380 Cause unknown. Possible work-around: "r.null setnull=nan"
381 If you encounter any problems (or solutions!) please contact the GRASS
382 Development Team.
383
385 g.region, m.proj, r.fillnulls, r.in.ascii, r.in.lidar, r3.in.xyz,
386 r.mapcalc, r.neighbors, r.out.xyz, r.to.rast3, r.to.vect, r.univar,
387 v.in.ascii, v.surf.rst
388 v.lidar.correction, v.lidar.edgedetection, v.lidar.growing, v.outlier,
390 v.surf.bspline
391 pv - The UNIX pipe viewer utility
393 Overview: Interpolation and Resampling in GRASS GIS
395
397 Hamish Bowman, Department of Marine Science, University of Otagom New
398 Zealand
399 Extended by Volker Wichmann to support the aggregate functions median,
400 percentile, skewness and trimmed mean.
401
403 Available at: r.in.xyz source code (history)
404 Main index | Raster index | Topics index | Keywords index | Graphical
406 index | Full index
407 © 2003-2020 GRASS Development Team, GRASS GIS 7.8.5 Reference Manual
409GRASS 7.8.5 r.in.xyz(1)