Surveying OpenStreetMap

CC-BY-SA 4.0 OSM CWG

Ahead of the Board Face to Face in Brussels we asked people across the project to share the most important topics facing OpenStreetMap. Thank you to every one of you who responded – 161 people in total! We want to share how we found this valuable and how it informed the meeting, some summary results and specific follow ups underway. Surveys are something with want to repeat – and we want help to implement with the best practice, and cover the right topics.

The value of surveying

You would think that the Board, seven people with presence in many different communities and communication channels, would know nearly everything happening in the project and what issues were top of mind. But with such an enormous scale of work and messaging, it’s impossible to form a cohesive picture. There are many people whose voices we don’t otherwise hear – due to language, the proliferation of channels, and communication style. We know that these 161 responses provide just one window into how we might improve community feedback loops. With a formal structure like the Board of Directors, with formal meetings and minutes and statements, it’s hard for everyone else to know we are listening.

OpenStreetMap is made up of everyone who takes part in it, and hearing each other’s voices in a coordinated way, on a regular basis, will help prioritize where work is needed and what actions to pursue. It ensures a standard way to engage. Surveying can help set up mechanisms to route issues to the right place, share pathways for OpenStreetMap members to contribute and address problems, and identify where the project as a whole needs more help to come up with answers. We can identify people and topics for more in depth follow up. This will only work and be useful if there is actual follow through and transparency on what we hear through surveys.

The results of this survey

Frederik summarized and classified each of the responses, to produce this summary list. This is just one way of classifying the responses – other Board members had similar breakdowns, but nothing as comprehensive. There weren’t tags for everything, and some responses were clearly not relevant. Most didn’t specify what they thought the Board could, but expressed generally things they’d like to see happen.

  • some form of Global Logic/Hostile Takeover concern
  • some concern about building a friendlier, better informed, more inclusive community (different users stressing different targets for inclusivity e.g. regions, countries, interest groups)
  • the suggestion that tagging rules need to be clearer, better documented, or a better process needs to exist
  • dissatisfaction with how the iD editor manages tagging decisions (largely a different group from the 13 as above)
  • need for better QA and vandalism detection
  • growing the project or membership in various ways
  • dissatisfaction with the website features, including tile speed
  • improve governance, improve Working Groups, better Conflict of Interest handling
  • hire developer and/or management staff
  • limit the ability of new mappers to contribute (and break things)
  • provide more/better education support, partner with education institutions
  • enforce attribution more
  • desire to have vector tiles on web site
  • make imports easier (and 2x control rogue imports better)
  • improve cartography
  • protect against commercial influence
  • overhaul wiki (but see tagging above, these could also mean tagging)
  • resource better imagery

We aren’t publishing the raw survey comments because we neglected to ask in the survey if it was ok to publish, or if the response should remain anonymous. This is something we will address in future surveys.

How we incorporated the responses into the meeting

We looked at the survey results after day 1 was done, so we had a full day to work already, directly with each other and all the ideas we each had brought the meeting. Saturday evening, we all received the results by email, and fair to say that all of us read every one. The next morning, we had spent time going round to everyone and hearing each other’s reflections, identifying where the survey input complemented work underway, as well as identify gaps in our scope.

Fair to say that building a more inclusive & healthy community, governance issues, hostile takeover, and growing the project were already top of mind. What surprised us were topics like tagging, software, and infrastructure issues. Of course we know these are top issues, but as the Board, it’s typically not been our role to get involved in such topics. Some of the concerns simply involved a rerouting problem – there are clear places to share feedback on things like the default cartography, and our plan is to share some of these items directly with those responsible. But for other issues like tagging, like iD, it’s more complicated and not clear how change is supposed to happen. There are “routing problems” in OSM and OSMF. We broadly think there may be a role for the Board in helping facilitate the community to find the way, but we’re not sure how it should work. In the mean time, we are having a few direct conversations to listen more.

We then later had half the Board take part in a 90 minute working session specifically on surveys (the other half worked on the takeover risk and governance changes topic). We generated ideas that make up much of this blog post – how are surveys valuable, what did we learn from this one, and how do we want to do surveys in the future.

Methods and topics for future surveys – we need your help

During our working session, we had to pull back several times from diving down the rabbit hole of methodology. We’ll start here with some top level design thoughts on how we see surveying at OSMF.

  • Keep a consistent pace of at least annually, and perhaps up to quarterly when topics are raised. These can be schedule at coordinated, quieter points along the “OSM Calendar” (prior to Annual General Meeting, Face-to-Face board meeting, State of the Map). Frequency also will help to gauge enduring issues, emerging issues, and one-offs.
  • Strike a balance of rigorous and achievable – we aren’t doing surveys for social science publication, but we do want to have a good standard of results we can trust.
  • Work collaboratively in building surveys with more experienced folks in OSM community, especially from academia/research. Gather input and refine questions from in person at SotMs, Working Groups, Chapters, Advisory Board, Regions.
  • Identify ownership of surveying processes in OSMF – for now the Board. This also means taking responsible stewardship of the submitted data.
  • Build a trusted process by demonstrating responsiveness and follow through to survey results. That means share summary results and actions by board/OSMF, as well as option for anonymously sharing raw results.
  • Reach a broad population, by a wide and documented set of outreach channels (mailing lists, forums, telegram, slacks, FB groups, etc). Also take distinct opportunities to have self selection surveys vs random selection where a specific population is chosen.
  • Surveys should not be burdensome on responders. Clearly say up front what will be asked, is it multiple choice or free text, and how much time is needed. Keep a baseline of background questions (like OSM experience and roles and location).
  • Make this a repeatable sound methodology. Means following a process step by actionable step.

Definitely opening thoughts. If you have experience in undertaking surveys, we’d love to talk to you about how to approach.

Finally, we brainstormed questions we’d like to ask on surveys. Broad topics were on our organizational development and governance, the technical roadmap and priorities, regional “nuances” in how OSM works, a survey directed to companies, and community engagement and health.

  • What are you doing that other communities can learn from?
  • Volunteering in OSM: what do you ideally want to do, and what do you need to do it?
  • Why are you in OSM? What was your best experience? What was your worst?
  • What are the different roles and kinds of contributions you have made?
  • Where do you communicate and why?
  • If you are not in OSMF, what would it take to convince you to join? What would you change?

What questions do you want to ask the OSM community? And, more importantly, how do you see survey results informing our collective next steps? Let us know either in the comments, on talk@openstreetmap.org or send an email to board@osmfoundation.org.

Want to help with methodology? Send us an email to board@osmfoundation.org with subject “Survey methodology help”, and we’ll let you know when we gather to start planning the next survey soon.

OpenStreetMap Foundation board


What is the OpenStreetMap Foundation
The OpenStreetMap Foundation is a not-for-profit organisation, formed in the UK to support the OpenStreetMap Project. It is dedicated to encouraging the growth, development and distribution of free geospatial data for anyone to use and share. The OpenStreetMap Foundation owns and maintains the infrastructure of the OpenStreetMap project, is financially supported by membership fees and donations, and organises the annual, international State of the Map conference. The OpenStreetMap Foundation has no full-time employees and it is supporting the OpenStreetMap project through the work of our volunteer Working Groups. Please consider becoming a member and read about our fee-waiver program.

What is OpenStreetMap
OpenStreetMap was founded in 2004 and is a international project to create a free map of the world. To do so, we, thousands of volunteers, collect data about roads, railways, rivers, forests, buildings and a lot more worldwide. Our map data can be downloaded for free by everyone and used for any purpose – including commercial usage. It is possible to produce your own maps which highlight certain features, to calculate routes etc. OpenStreetMap is increasingly used when one needs maps which can be very quickly, or easily, updated.