Category Archives: Software

Software

Whitebox GAT Usage

Check out Prof. John Lindsay’s open-access GIS project – Whitebox GAT. It is an outstanding GIS software package with a lot of advanced geoprocessing tools!
http://www.uoguelph.ca/~hydrogeo/Whitebox/

Whitebox Geospatial Analysis Tools

It is difficult to get reliable information about the usage of open-source software like Whitebox GAT. With the recent release of Whitebox 3.4 ‘Montreal’, I decided to undertake an analytical exercise to try and figure out where Whitebox GAT is being used based on some download information that I had accumulating in my inbox. This work updates a previous survey that I carried out a number of years back, shortly after the 1.0 release. Examining nearly 21,000 downloads of the software, I discovered the following:

  • Whitebox GAT has been downloaded in at least 178 counties worldwide, with the top 10 countries, in terms of number of downloads, being the United States, Canada, India, Italy, South Africa, the United Kingdom, Germany, Brazil, Australia, and Spain.
  • Whitebox GAT has been downloaded in 5149 cities around the globe. The map below shows each city in which Whitebox GAT was downloaded at least one time over the…

View original post 151 more words

Setting up Anaconda, PySAL with ArcGIS Python environment

This tutorial shows you how to set up conda environment to work with ArcGIS 10.4 and ArcGIS Pro 1.3. You can download a pdf copy of the tutorial with screenshots HERE. At the 2016 Esri International User Conference in San Diego last month, Esri released ArcGIS Pro 1.3, which can now use conda for packaging Python libraries. This allows support of python under multiple Python environments. You no longer need to install a separate Python install to get the full Python capability with ArcGIS as you did with past versions.

Workflow to set up Anaconda with ArcGIS 10.4

  • Install Anaconda without fouling the Windows environment (paths, registry) to break Esri’s python stack
  • Configure Anaconda with the particular add-ons you want, and
  • Configure ArcGIS’s Python so that it is aware of the appropriate Anaconda content.

1) Install Anaconda for All Users

  1. Go to http://continuum.io/downloads
  2. Download the 32-bit version of Anaconda (Python 2.7)
  3. In the install dialogs:
    • Select install for All Users
    • conda01
    • Install to a folder by default (C:\Anaconda2)
    • conda02
    • IMPORTANT: To avoid breaking ArcGIS (or other software), uncheck the checkboxes (a) make Anaconda the default Python and (b) add Anaconda’s Python to the PATH.
    • conda03
  4. Go to Start > All Programs(apps) > Anaconda2(32-bit) > Anaconda Prompt. Right click, run as administrator

2) Configure an Anaconda environment for use with ArcGIS

  1. Find the versions of numpy and matplotlib ArcGIS is using.

Open ArcMap and its Python window, and enter these commands:

    • >>> import sys, numpy, matplotlib
    • >>> print(sys.version, numpy.__version__, matplotlib.__version__)
    • (‘2.7.10 (default, May 23 2015, 09:40:32) [MSC v.1500 32 bit (Intel)]’, ‘1.9.2’, ‘1.4.3’)
    • conda04
  1. Create an Anaconda environment that is compatible with ArcGIS
    • Get to the Anaconda Command Prompt (Start > All Programs(apps) > Anaconda2(32-bit), pick “Anaconda Prompt“), Right click, Run as Administrator.
    • Type (depending on ArcGIS version, I am using ArcGIS 10.4 as an example here):
    • “conda create -n arc104 python=2.7.10 numpy=1.9.2 matplotlib=1.4.3 pyparsing xlrd xlwt pandas scipy ipython ipython-notebook ipython-qtconsole”
    • conda05
    • Enter y to proceed.
    • Anaconda’s conda command will then set up an environment subdirectory, ex: ” C:\Anaconda2\envs\arc104 “, installing the downloaded packages into it.
    • conda06
  2. Test the virtual environment
    • At the Anaconda Command Prompt, type: activate arc104
    • Type: conda list. You can see the list of packages installed.conda07
  3. Add more packages
    • You can add more packages using conda install, but make sure you specify version numbers for these that won’t change the environment’s version of python or numpy (or ArcGIS will not be able to use that environment anymore).
    • Let’s add the Python Spatial Analysis Library (pysal) module.
    • Type the following command at the Anaconda Prompt:”conda install -n arc104 python=2.7.10 numpy=1.9.2 matplotlib=1.4.3 pysal”
    • conda08

3) Configure ArcGIS to see Anaconda and vice versa

  1. Anaconda Python to ArcPy
    • Copy the Desktop10.4.pth file to the Anaconda environment site-packages folder:
    • From: C:\Python27\ArcGIS10.4\Lib\site-packages\Desktop10.4.pth
    • To: C:\Anaconda2\envs\arc104\Lib\site-packages\Desktop10.4.pth
  2. Arcpy to Anaconda Python
    • Create a zconda.pth (path) file with the content “C:\Anaconda\envs\arc104\lib\site-packages” in it.
    • Then copy zconda.pth to C:\Python27\ArcGIS10.4\Lib\site-packages
  3. Testing in ArcMap
    • As a regular user, start ArcMap, open the Python window
    • type “import pysal”
    • type “pysal.” A popup menu with a list of pysal-provided functions is a pretty good sign the installation succeeded.
    • conda09
  4. Testing in PyCharm
    • Start PyCharm, in File\Settings…, choose Project then Project Interpreter
    • Ignore the drop down list for Project Interpreter, and click the cog button to Add Local, and in the file browser pick C:\Anaconda2\envs\arc104\python.exe
    • conda10
    • To run your script, right click it in the Project window, and choose either Run or Debug
    • Restart PyCharm for the Python Console to use the arc104 environment.
    • conda11

Workflow to set up Anaconda with ArcGIS Pro 1.3

1) Create an Anaconda environment that is compatible with ArcGIS Pro

  • Copy the folder arcgispro-py3 from C:\Program Files\ArcGIS\Pro\bin\Python\envs and paste to C:\Anaconda2\envs
  • Rename the copied folder arcgispro-py3 in C:\Anaconda2\envs to arcpro

2) Test the virtual environment

  • At the Anaconda Command Prompt, type: activate arcpro
  • Type: conda list. You can see the list of packages installed

3) Add more packages

  • Let’s add the Python Spatial Analysis Library (pysal) module.
  • Type the following command at the Anaconda Prompt:”conda install pysal”

4) Configure ArcGIS to see Anaconda and vice versa

  • Arcpy to Anaconda Python
    • Create a zconda.pth (path) file with the content “C:\Anaconda2\envs\arcpro\lib\site-packages” in it.
    • Then Copy zconda.pth to C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages
  • Testing in ArcGIS Pro
    • Start ArcGIS Pro, open the Python window
    • type “import pysal”
    • type “pysal.” A popup menu with a list of pysal-provided functions is a pretty good sign the install succeeded.
    • conda12
    • conda13
  •  Testing in PyCharm
    • Start PyCharm, in File\Settings…, choose Project then Project Interpreter
    • Ignore the drop down list for Project Interpreter, and click the cog button to Add Local, and in the file browser pick C:\Anaconda2\envs\arcpro\python.exe
    • conda14
    • To run your script, right click it in the Project window, and choose either Run or Debug
    • Restart PyCharm for the Python Console to use the arcpro environment.

References

  1. USGS: https://goo.gl/xd6xz2
  2. Esri: https://goo.gl/tYGHrw
  3. GeoNet: https://goo.gl/mTLWMG
  4. UC-Davis: http://goo.gl/3bdbwz

Esri Releases Drone2Map for ArcGIS

Drone2Map for ArcGIS, released on February 24 by Esri, is a stand-alone desktop app for processing imagery collected by drones. Check out the Drone2Map FAQ and an interesting presentation (Working With Drone Data In ArcGIS) by Tony Mason of Esri. Interested users can visit esri.com/drone2map for more information.

Q: Is Drone2Map for ArcGIS going to be an ArcGIS Extension?
A: No. It is a stand-alone 64-bit Windows desktop app that will run alongside ArcMap and ArcGIS Pro.

Q: What does Drone2Map for ArcGIS do?
A: Drone2Map for ArcGIS is a desktop app that turns raw still imagery from drones into  stunning information products in ArcGIS. Now, with drone hardware becoming more accessible, you can create 2D and 3D maps of features and areas.

Q: Can Drone2Map for ArcGIS be used to make 3D models?
A: Yes, Drone2Map for ArcGIS will produce 3D colorized point clouds in LAS format as well as 3D textured meshes for use in ArcGIS Desktop and Web Apps.

Q: Does the Drone2Map for ArcGIS work only with a specific type of drone?
A: Drone2Map for ArcGIS is designed to be generic for all drones. What is important is that the drone collects certain types of metadata. At a very minimum, this metadata needs to include Latitude, Longitude, and Altitude. The addition of orientation, focal length and pixel size of the sensor will greatly improve results. Many commercially available drones have this capability and automatically add this information to the image metadata.

Snap3

Snap4

Tutorials for processing Sentinel-1data

I just started exploring Sentinel-1 SAR data for my research on wetlands and water resources. Here are some useful resources I found:

About the Sentinel-1 mission:

Blog posts:

Advanced training course on the use of Sentinel-1 SAR data:

Sentinel-1 data analysis using PCI Geomatica:

Synthetic Aperture Radar: Of Bats and Flying Pianos:

  • An amusing introduction to radar remote sensing from satellites, with the concept of “range Doppler” image formation described using entertaining audio-video animations.


ArcGIS 3D LiDAR Toolset

ESRI has released a new version of 3D LiDAR toolset, which was designed to extend the LiDAR capabilities of ArcGIS Desktop. It can be downloaded from : http://www.arcgis.com/home/item.html?id=fe221371b77940749ff96e90f2de3d10

cwduyyhweaau4fdFunctionalities of the LiDAR toolset:
  • Classify ground*, building, vegetation, and noise points
  • Extract building footprint approximations
  • Clip LAS files*
  • Improve QA/QC processes with lidar data:
    • Evaluate LAS files for errors through the CheckLAS utility
    • Export LAS file header information
    • Define the spatial reference of LAS files with missing/incorrect information*
    • Project LAS files to desired coordinate systems*
    • Evaluate coverage of overlaps in lidar scans
    • Rearrange LAS files to optimize data access I/O*
  • Optimize lidar data for operational use and rapid access through the compressed ZLAS format
  • Evaluate Z statistics with advanced height metrics*
  • Analyze the proximity of LAS points to 3D features**
  • Convert lidar data between various data formats
  • Create tiled raster derivatives
Analysis & Data Management of 3D Features & Surfaces
  • Correct the Z value of a multipatch model so that it “sits” on the ground
  • Create a point skymap of sun positions for visualization and solar analysis workflows
  • Simplify dense, 3D breaklines to support scalability in TIN-based surface modeling*
  • Integrate a design surface, such as one created using the Grading tool, into a base TIN
  • Export a TIN to LandXML for use in 3rd party applications
  • Cross sections of a multipatch can be used with the Intersect 3D tool to:
    • Generate contours in 3D space that capture cliff overhangs
    • Determine a 3D model’s footprint at different heights
    • Generate sightlines for visibility analysis

FREE GIS Software for Mapping

Whitebox GAT (Geospatial Analysis Toolbox) is my favorite open source, free GIS software!  Come join the Listserv and follow the Whitebox GAT blog by Dr. John Lindsay.  Happy Geoprocessing!

Source: http://gisgeography.com/free-gis-software/

You don’t have to pay an arm and a leg to map the world.

With a shoestring budget of 0$, you can do it all with free GIS software.

As you’re about to see, there’s a bucket load of free GIS software packages available with the capability to:

  • Perform hundreds of advanced GIS processing tasks.
  • Generate stunning cartography and mapping products.
  • Manage your company’s geospatial assets efficiently.

The best part is:

These free GIS software mapping options give you enough firepower to get the job done as if you’re working with commercial GIS software.

I personally tested 13 of the top free mapping software packages online.

Read this jam-packed guide to open source GIS:

1. QGIS – Formerly Quantum GIS

QGIS (Quantum GIS)

With striking similarities and even upgrades to ArcGIS in certain areas, QGIS is undoubtedly the #1 free GIS software package.

QGIS is jam-packed with hidden gems at your fingertips. Automate map production, process geospatial data, and generate drool-worthy cartographic figures.

There’s no other free maping software on this list that lets you map like a rock star than QGIS.

QGIS Plugins boost this mapping software into a state of epicness. If the tool doesn’t exist, search for plugin developed by the QGIS community.

Volunteer effort is key to its success. The QGIS Stack Exchangesupport is impressively great.

You’d be insane not to download the free GIS software QGIS.

Read More: 27 Differences Between ArcGIS and QGIS – The Most Epic GIS Software Battle in GIS History

2. Whitebox GAT

WhiteBox GAT

Yes, Whitebox GAT (Geospatial Analysis Toolbox) is #2 on my list of open source, free GIS software.

Unbelievably, Whitebox GAT has only been around since 2009 because it feels so fine-tuned when you see it in action.

There’s a hydrology theme around Whitebox GAT. It actually replaced Terrain Analysis System (TAS) – a tool for hydro-geomorphic applications.

Whitebox GAT is really a full-blown open-access GIS and remote sensing software package.

Where it shines is LIDAR!

With no barriers, Whitebox GAT is the swiss-army knife of LiDAR processing.

The LiDAR toolbox is a life-saver. LAS to shapefile is an insanely useful tool. You may need a Java update to go in full throttle though.

The cartographic mapping software tools are primitive compared to QGIS.

But overall Whitebox GAT is solid with over 410 tools to clip, convert, analyze, manage, buffer and extract geospatial information.

I find it amazing this free GIS software almost goes unheard of in the GIS industry.

Get more useful knowledge from the Whitebox GAT Open Source Blog.

3. GRASS GIS

GRASS GIS Desktop

GRASS GIS (Geographic Resources Analysis Support System) was developed by the US Army Corps of Engineers as a tool for land management and environmental planning.

It has evolved into a free GIS software option for different areas of study.

Academia, environment consultants and government agencies (NASA, NOAA, USDA and USGS) use GRASS GIS because of its intuitive GUI and its reliability.

It has over 350 rock-solid vector and raster manipulation tools.

Not awfully useful in cartographic design, GRASS GIS excels primarily as a free GIS software option for analysis, image processing, digital terrain manipulation and statistics.

4. SAGA GIS

SAGA GIS

SAGA GIS (System for Automated Geoscientific Analyses) is one of the classics in the world of free GIS software.

It started out primarily for terrain analysis such as hillshading, watershed extraction and visibility analysis.

Now, SAGA GIS is a powerhouse because it delivers a fast growing set of geoscientific methods to the geoscientific community.

Enable multiple windows to lay out all your analysis (map, histograms, scatter plots, attributes, etc). It provides both a user-friendly GUI and API.

It’s not particularly useful in cartography. The lack of scale bars, north arrows symbology and grids are telltale signs of this.

Overall, it’s quick, reliable and accurate. Consider SAGA GIS a prime choice for environmental modeling and other applications.

Read more at http://gisgeography.com/free-gis-software/

Access Landsat Imagery in ArcGIS Pro

Related post: Landsat 8 data on Amazon AWS

Here’s how you can access the Landsat archive which is hosted on Amazon through ArcGIS Pro.

  1. Open the Projects Pane
  2. Click on Portal and then on the cloud icon on the far right to see everything that’s on your portal.
  3. Search for “Multispectral Landsat” in the search bar.
  4. Drag and drop.

To take advantage of the archive, we’ve designed a number of pre-defined band combinations and indices. You can find these from the Data tab, under Processing Templates.

Play around with these. Here’s some cool stuff that I found poking around in Alaska for a few minutes.

Read the rest of this entry

Whitebox GAT’s new website and other developments

Check out Prof. John Lindsay’s new Whitebox GAT website and the GoSpatial. A lot of cool and innovative stuff!!

“GoSpatial is a command-line interface program for analyzing and manipulating geospatial data. It has been developed using the Go programming language and is compiled to native code. The project is experimental and is intended to provide additional analytical support for the Whitebox Geospatial Analysis Tools open-source GIS software. GoSpatial can however be run completely independent of any other software and is run from a single self-contained executable file.”

Whitebox Geospatial Analysis Tools

There are a few exciting announcements related to new developments on the Whitebox GAT front. The first is that due to changes in the Google Code practices (it has become ‘read only’ and is no longer accepting new code commits), the Whitebox project has moved its source code repository to GitHub. I think that this will eventually make for improved source code management, although there may be some initial transition issues that we’ll need to work past. Some of the documentation will have to be updated to reflect this change.

The second announcement, which I am most excited about, is that I have finally found the time to update the Whitebox GAT website. There is a fresh new and more professional look to the site. I hope you enjoy the new webpage and as always, feedback is welcome. What would you like to see added or changed?

Lastly…

View original post 88 more words

2015 Esri User Conference Plenary Session Videos Available

The 2015 ESRI International User Conference is happening now in San Diego, California with 16,000 attendees. Some of the new features released on Monday’s plenary sessions include:

My personal favorite is the R & ArcGIS. More information about this can be found in another blog at https://gisday.wordpress.com/2015/07/21/r-bridge-for-arcgis/.

The following videos from the Plenary Session on Monday, July 20, 2015, are now available for viewing at http://video.esri.com/series/250/2015-esri-user-conference-plenary-_dash_-watch-the-entire-day:

• Keynote from Jack Dangermond
• R&D at Esri
• Southwest Florida Water Management District – A Mission-Critical Approach to Water
• Hartsfield-Jackson Atlanta International Airport – Web GIS Delivers Safety and Efficiency
• Beck’s Hybrids – Feeding the World: Precision Agriculture Simplified
• State of Victoria, Australia, Government – Proactive Rejuvenation: Ignite Your Potential
• World Health Organization – The Battle against Ebola and Polio
• National Geographic Society – Understanding Geography
• Mentoring the Next Generation – Connecting GIS with Education
• Keynote from Martin O’Malley – Smart Government: Reduce the Distance between People

Check out here for some pictures I took at the conference.

R-ArcGIS

R-bridge for ArcGIS

Source: http://blogs.esri.com/esri/esri-insider/2015/07/20/building-a-bridge-to-the-r-community/

R-bridge for ArcGIS: https://github.com/R-ArcGIS/r-bridge

R-ArcGIS: https://r-arcgis.github.io/

Today at the Esri User Conference in San Diego, Esri announced a new initiative to build a collaborative community for R and ArcGIS users.

Esri has been teaching and promoting integration with R at the User Conference and Developer Summit for several years. During this time we have seen significant increase in interest, and received useful feedback from our ArcGIS users and R users about a variety of needs and techniques for integrating ArcGIS and R. Based upon this feedback, we are working with ArcGIS and R users to develop a community to promote learning, sharing, and collaboration. This community will include a repository of free, open source, R scripts, geoprocessing tools, and tutorials.

I recently sat down with Steve Kopp, Senior Product Engineer on the spatial analysis team, and Dawn Wright, Esri’s Chief Scientist to talk about what this focus on building a bridge to the R community means for ArcGIS users and other users of R.


Matt Artz:
What is R?

Steve Kopp: R (aka the R Project for Statistical Computing) is an extremely popular and the fastest growing environment for statistical computing. In addition to the core R software, it includes a more than 6,000 community-contributed packages for solving a wide range of statistical problems, including a variety of spatial statistical data analysis methods.

Dawn Wright:  R is widely used by environmental scientists of all stripes, as well as statisticians. Since R has limited data management and mapping capabilities, many of our users find good synergy in using R and ArcGIS together.


Matt Artz:
Does the ArcGIS community use R today?

Steve Kopp: Yes, R has become very popular in the ArcGIS community over the last several years.  Many in our user community have been asking for a mix of its functionality with our own, as well as better code-sharing interaction with the R community.

Dawn Wright: A great example from the marine ecology community is Duke University’s Marine Geospatial Ecology Tools, where they have already long since moved forward with integrating R and ArcGIS for Desktop for some time.


Matt Artz:
What is the R – ArcGIS Bridge?

Steve Kopp: This is a free, open source R package which allows ArcGIS and R to dynamically access data without creating intermediate files on disk.


Matt Artz:
Why did Esri build the R – ArcGIS Bridge? 

Steve Kopp: It was built for three reasons: to improve the performance and scalability of projects which combine R and ArcGIS; to create a developer experience that was simple and familiar to the R user; and to enable an end-user experience that is familiar to the ArcGIS user.

Dawn Wright: The bottom line is that this project is about helping our user community and the R user community to be more successful combining these technologies.


Matt Artz:
So is this initiative just some software code?

Steve Kopp: No, the R – ArcGIS Bridge is simply some enabling technology, the real effort and value of the R – ArcGIS Community initiative will be in the development and sharing of useful tools, tutorials, and tips. It’s a free, open source library which makes it fast and easy to move data between ArcGIS and R, and additional work which makes it possible to run an R script from an ArcGIS geoprocessing tool.

Dawn Wright: This community will be important and useful for R users who need to access ArcGIS data, for ArcGIS users who need to access R analysis capabilities from ArcGIS, and for developers who are familiar with both ArcGIS and R who want to build integrated tools or applications to share with the community.

Steve Kopp: The community of tools will be user developed and user driven. Esri will develop a few sample toolboxes and tutorials, but our primary interest is to facilitate the community and help them build what they find useful.


Matt Artz:
How do you see the ArcGIS community using the R – ArcGIS Bridge?  What does it give them that they don’t have today? 

Steve Kopp: The R – ArcGIS Bridge allows developers with experience with R and ArcGIS to create custom tools and toolboxes that integrate ArcGIS and R, both for their own use, and for building toolboxes to share with others both within their organization and with other ArcGIS users.

Dawn Wright: R developers can quickly access ArcGIS datasets from within R, save R results back to ArcGIS datasets and tables, and easily convert between ArcGIS datasets and their equivalent representations in R.

Steve Kopp: It allows our users to integrate R into their workflows, without necessarily learning the R programming language directly.


Matt Artz:
What about the R user who doesn’t use ArcGIS?

Steve Kopp: It’s not uncommon in an organization for a non-GIS person to need to be able to work with GIS data; for these people, they will be able to use the bridge to directly access ArcGIS data without creating intermediate shapefiles or tables, and without needing to know any ArcGIS.


Matt Artz:
How can people start using the R Bridge? 

Steve Kopp:  The R – ArcGIS community samples, tutorials, and bridge are all part of a public GitHub community site similar to other Esri open source projects. And if you happen to be at the Esri User Conference in San Diego this week, this project will be discussed as part of a workshop on Wednesday.

– See more at: http://blogs.esri.com/esri/esri-insider/2015/07/20/building-a-bridge-to-the-r-community/#sthash.0lSJ6kGl.dpuf

 

REDD+ for the Guiana Shield

Technical Cooperation Project

LidarBlog.com

Dr. Qiusheng Wu @ SUNY Binghamton

Writing Science

How to write papers that get cited and proposals that get funded

GIS In Ecology

Providing Training, Advice And Consultation On The Use Of GIS In Ecology

metrhispanic

On cities, land, ...

GeoAcademy

Open GIS: No Bounds

Scientia Plus Conscientia

Thoughts on Science and Nature

r4hydrology

Learning hydrology with R

Karl Hennermann

GIS at the University of Manchester

GIS and Science

Applications of geospatial technology for scientific research and understanding.

GISblog.com

GIS, Mapping, Remote Sensing, Geodata, Geospatial news

Whitebox Geospatial Analysis Tools

Open-source GIS development and spatial analysis with Whitebox GAT

TopoToolbox

MATLAB-based software for topographic analysis

Anything Geospatial – AnyGeo

Dr. Qiusheng Wu @ SUNY Binghamton

GeospatialPython.com

Dr. Qiusheng Wu @ SUNY Binghamton

Another GIS Blog

Dr. Qiusheng Wu @ SUNY Binghamton

ArcPy Café

Get all your ArcGIS Python Recipes here!

Planet Python

Dr. Qiusheng Wu @ SUNY Binghamton