Updates as of 2018-01-16

• Magicarpic PMed me details of a Ukraine EX raid which had a leisure=park polygon created 15th July 2016. This sets a new earliest date.
  
• MzRed found EX raids corresponding with landuse=grassland
  
• Montagemz found an EX raid corresponding with landuse=farmyard
  
• 0Geert0 found an EX raid corresponding with natural=heath
  
• Groschenprinz05 found an EX raid corresponding with landuse=vineyard
  
• Flitzer09 found an EX raid corresponding with landuse=farmland
  
• GizzlySGD found an EX raid corresponding with landuse=orchard
  
• Given the complete lack of evidence for leisure=nature_reserve even being a nesting tag, I have removed it from the query. It is a much more common tag than orchard, vineyard, heath, etc., yet we still haven't had a single person with a leisure=nature_reserve EX raid. (Edit 2018-01-27: Fixed an issue where I had left part of the leisure=nature_reserve code in)
  
• Updated overpass-turbo query incorporating these tags: http://overpass-turbo.eu/s/vs3
  

 

1.0 Background

Last week I posted an analysis of 49 EX raid locations in Western Australia. The key findings were:

• 100% of EX raids from December, 2017 in Western Australia corresponded with OpenStreetMap tags associated with nests (e.g. leisure=park, landuse=recreation_ground)
• In three instances the gym was slightly outside the polygon of the park from OSM, but if the park was overlayed with level 20 s2 cells then the gym was inside these cells. I proposed that Niantic stores information about parks as level 20 s2 cells and this is why these three gyms were eligible for EX raids.

 
There still remained some follow up questions:

• When is the OSM data for EX raids sourced from?
• What is the full range of OSM tags that can lead to EX raids?
• Was the level 20 s2 cell overlap just a coincidence with those three gyms from Western Australia, or could this be confirmed with other EX raid gyms?

 
To answer those questions I have spent a good chunk of the last week performing a larger analysis of EX raid locations from around the world. The following post describes my key findings.
 
Note #1: Throughout this article where I refer to ‘parks’, I am generally referring to the entire collection of tags on OSM which lead to nests, including landuse=recreation_ground, leisure=garden, landuse=grass, etc. If I want to refer to leisure=park polygons specifically then I will use that term.
 
Note #2: This research will only deal with the crietria that allows a non-sponsored gym to be eligible to host EX raids. It does not deal with the selection mechanism (i.e. which gym is chosen each week).
 

2.0 Methodology

2.1 Data collection

Data was collected about EX raids from locations worldwide. This was sourced through collections previously posted on Reddit, local Discord groups and data sent to me as response to a thread I posted a few days ago (both publically and via PMs). The following table shows the number of gym locations from each country/region. The distribution of gym locations is also shown in the following graphic: https://i.imgur.com/QnL1vPe.png
 

Region	Number of EX Raid Locations
Canada	248
Australia	224
Hong Kong	140
Singapore	134
United States	130
United Kingdom	61
France	52
Brazil	29
Germany	22
Belgium	15
United Arab Emirates	5
Total	1060

 
The dataset included non-sponsored gyms only. Dates of EX raids ranged from 30 September, 2017 to 25 December, 2017 (encompassing all non-sponsored EX raid dates prior to the latest January wave). 1010 out of the 1060 were for dates ranged from 11 November, 2017 to 25 December, 2017.
 

2.2 Analysis of parks

Data was exported from OSM using overpass-turbo. A list of the exported tags is available here: http://overpass-turbo.eu/s/vs3. Data was exported from one-month intervals (1 Mar 2016, 1 Apr 2016, 1 May 2016 … 1 Jan 2017). Data was also exported from the date of the latest nest update (22 Jan 2017), the date of the visual map data in the game (13 Aug 2017) and current (1 Jan 2018).
 
I analysed gyms using each month’s data via a point-in-polygon method using R (see description here). For gyms which failed the point-in-polygon test I checked level 20 s2 cells using osmcoverer by MzRed which was being developed as I was completing my analysis. (At that stage osmcoverer did not have gym marker capability or I would have just used osmcoverer instead of R). For any gyms which showed a difference in the monthly data (e.g. fell inside a park polygon using data from Jul 2016 but did not fall inside a polygon using data from Jun 2016) the data was manually investigated on OSM to check for the specific date resulting in that change.
 

3.0 Findings

3.1: Changes in EX raid eligibility criteria over time

My previous analysis only included data from December 2017, whereas now I had collected data dating back to 30 September, 2017. After analysing which gyms fell inside or outside parks, it became clear that Niantic changed the way that non-sponsored gyms were chosen for EX Raids between the 20 October EX raid and the 11 November EX raid.
 

EX Raid Date	% Parks	% Non-Parks
30 Sep/1 Oct	41%	59%
20 October	27%	73%
11 November	100%	0%
18 November	100%	0%
26 November	100%	0%
3 December	100%	0%
11 December	100%	0%
18 December	100%	0%
25 December	100%	0%

 
Since the change, 100% of non-sponsored gyms can be explained using OSM tags, without exception. This could also explain why there was a three-week break in non-sponsored raids; during this time Niantic was modifying their selection algorithm to target parks.
 
Future research will need to pay attention to EX raid release dates, as the difference between pre-November and post-November raids does affect analysis. A number of comments in my previous thread provided examples of EX raids not in parks, however follow ups confirmed that these invites were for raids in SeptembeOctober. Additionally, there is no guarantee that Niantic will not change the formula again.
 
Implications: Unfortunately, if your town's park areas were not adequately mapped in OSM and you don’t have sponsored gyms then it appears you are out of luck for EX raids. Despite having over 1000 EX raid locations from November and December, not a single instance was found of a non-sponsored EX raid occurring without a corresponding OSM tag.
 

3.2: Use of level 20 s2 cells to determine if gyms are inside parks

In my previous thread I established the idea that the boundaries of parks are defined by level 20 s2 cells. This was used to account for three gyms in Western Australia which had EX raids despite falling just outside the polygon of the parks on OSM.
 
I can confirm after looking at the expanded dataset that this was not a coincidence. In total there were 40 gyms (out of 1060) which fit this circumstance: they could only be explained based on s2 cell overlap. A few examples from these 40 gyms are shown below (maps generated using osmcoverer by MzRed).
 

Gym	Latitude	Longitude	Map of s2 cells
Pavilion of Yew Tee Park	1.397505	103.744281	link
Circle of Pillars	1.345777	103.693723	link
Alumni Field Commemoration	43.473968	-80.525313	link

 
I am confident enough to say that this confirms that level 20 cells are used to determine whether gyms are in parks.
 
Implications: Previous tools which have looked solely at whether gyms lay within polygons might have excluded a small proportion of eligible gyms. Based on this dataset, ~4% of gyms would not have been predicted to be EX raid eligible if level 20 s2 cells were not considered.
 

3.3: Date range for OSM data

Establishing the earliest date of OSM data is straightforward: look for locations that have had EX raids, and then look for when the corresponding OSM feature was created. There are three gyms which were able to place the earliest starting date in July, 2016:
 

Gym	Latitude	Longitude	Creation date of corresponding tag
Medford Statue of Liberty	42.323083	-122.876866	6 July 2016 (OSM link)
上帝古廟	22.326202	114.185189	9 July 2016 (OSM link)
九龍城立方體地標	22.327121	114.185292	9 July 2016 (OSM link)

 
Finding the oldest possible date of the data is expected to be more challenging. It requires an EX raid to be held at a place that has had the tag removed, either because the park itself has been removed or because the park was plotted incorrectly and has been corrected. Finding the oldest possible date for the data therefore depends on luck.
 

Gym	Latitude	Longitude	Removal/modification date of corresponding tag
Towers Baptist Church	49.142106	-123.109827	6 January 2017 (OSM link)
[Golf course gym]	[Withheld]	[Withheld]	17 November 2016
Jin Fu Gong Temple	1.340732	103.690463	31 August 2016 (OSM link)
南洋公园	1.340382	103.690883	31 August 2016 (OSM link)

 
LimboMon had already discovered a possible date of August, 2016 for the OSM data based on a Singapore gym which had landuse=greenfield. At the time it was difficult to assert this with confidence for two reasons: firstly,landuse=greenfield was a tag that had not previously been linked to nests; secondly, not enough data had been collected to clearly establish that 100% of EX raids correspond to OSM tags.
 
Using a separate line of data I have been able to prove that the EX raid data pre-dates the nest data (January 22, 2017). Towers Baptist Church in Canada (49.142106, -123.109827) held an EX raid on 18th December, 2017. Until 6 January 2017 the gym was covered by a leisure=park polygon which covered the church, however after this date a user modified the park polygon to exclude the church building. Compare the level 20 overlay before 6 Jan 2017 and after 6 Jan 2017.
 
In my last analysis thread, DrKillerZA provided me coordinates of an EX raid in South Africa which was at a golf course. The polygon for the golf course had been modified in November 2016 which removed the gym from the range of the golf course (compare the level 20 overlay before 17 Nov 2016 and after 17 Nov 2016). I was recently able to confirm that the EX raid at this gym occurred on December 18, which means it was subject to the parks requirement for non-sponsored EX raids, and hence the OSM data must be prior to 17th November 2016.
 
Finally, I can also reconfirm the Singapore data. In addition to Jin Fu Gong Temple (1.340732, 103.690463), which held an EX raid on 11 November 2017, there is now also 南洋公园 (1.340382, 103.690883) which held an EX raid on 11 December, 2017. Both of these gyms were in range of a landuse=greenfield tag that was removed 31 August 2016 when a nursing home was built on the site. There are no other nearby tags which could trigger EX raid status, even when considering level 20 s2 cells.
 
Based on the above evidence we can conclude that the date range for the OSM data used for EX raids is from somewhere between 9th July 2016 and 31st August 2016. It is likely more towards the July date, given that park additions are more common than park removals and therefore the earliest possible OSM date is easier to locate than the latest possible OSM date.
 
Implications: OSM received a flurry of attention in Dec 2016-Jan 2017 on Silph Road when the links between OSM and Pokemon Go became increasingly clear. This led to a lot of players checking out OSM and mapping parks in their area to attempt to get nests. Unfortunately, the EX raid data pre-dates this. Players may have been targeting gyms incorrectly believing that they can host EX raids when they actually cannot.
 

3.4: OSM tags associated with EX Raids

My initial study showed that leisure=park and landuse=recreation_ground were the two most common tags associated with non-sponsored EX raids. I wanted to use the larger dataset to explore which additional tags can be proven to lead to EX raids. For this stage of the analysis, I first removed every EX raid that could be explained using leisure=park or landuse=recreation_ground (including those which were covered by level 20 s2 cells). This accounted for 912/1010 gyms. For the remaining gyms, I manually checked the corresponding way(s) on OSM.
 
The following table shows the raw number of gyms which SOLELY matched each tag and no other.
 

OSM tag	Matching gyms	Example gym
landuse=grass	36	Church of Canada (45.493796, -73.575442) (OSM)
leisure=playground	16	Kam Ying Fountain (22.422243, 114.23617) (OSM)
leisure=garden	15	Confucious Statue (29.721873, -95.387988) (OSM)
leisure=recreation_ground	8	Dancer and the Clock (1.348024, 103.756122) (OSM)
leisure=pitch	8	Don Dawson Oval (-33.904022, 150.933424) (OSM)
landuse=meadow	7	CSI Campus Centre (40.601612, -74.148523) (OSM)
leisure=golf_course	2	Casino Grove Entry (-38.118564, 145.25032) (OSM)
landuse=greenfield	2	Jin Fu Gong Temple (1.340732, 103.690463) (OSM)
natural=scrub	1	Autobahnkirche Ruhr (51.496269, 7.188764) (OSM)
landuse=farmyard	1*	Parkgutt/Gutt med sydvest i bronse (59.895986,10.812309) (OSM)
natural=grassland	1*	Kukkiva Maisemapelto (61.475845,23.824455) (OSM)
boundary=physiogeographical	0	-
boundary=nature_reserve	0	-
leisure=nature_reserve	0	-
natural=heath	0	-
natural=moor	0	-
landuse=farmland	0	-
landuse=orchard	0	-
landuse=vineyard	0	-

 
Edit 1: Added landuse=farmyard example mentioned here.
Edit 2: Some people have asked about parks, etc. tagged as relations instead of ways (rel[leisure=park] instead of way[leisure=park]). 100% of the gyms I tested could be explained only using ways. If you find an example of an EX raid gym that requires relations to explain it, let me know.
Edit 3: MzRed provided me two examples of gyms which can only be explained using natural=grassland. I have added this to the table and have modified the overpass-turbo query.
 
This confirms the ability of 11 tags (including leisure=park and landuse=recreational_ground) to generate EX raids. The lack of results for the last six OSM tags does not mean that they cannot spawn EX raids. These tags may be rare, may not contain gyms, or may not be in well-frequented areas (or possibly all three).
 
Implications: It appears that most tags that lead to nests can lead to EX raids, however keep in mind that EX raids use older data than current nests so there is not a 1:1 correlation.
 

4.0 Application of new findings

4.1 Western Australian gyms: A case study

I have a copy of location details for all gyms in Perth, Mandurah and Bunbury, Western Australia. Using these I had previously used the point-in-polygon method using OSM data from 22 January, 2017 to create a map of predicted EX raid gyms. However the latest studies have shown that I would have had some false positives (gyms matched as positive when due to older-than-expected data they should’ve been negative) and some false negatives (gyms outside polygons but inside level 20 s2 cells).
 
I re-ran the gym prediction tool on my Perth gyms dataset using OSM data from different dates, and with and without accounting for s2 cells, resulting in the following numbers:
 

OSM source date	Gyms using OSM polygon method (incorrect)	Gyms using s2 cells method (correct)
22 Jan 2017	956	1082
31 Aug 2016	932	1058
9 Jul 2016	928	1054

 
Overall there was a net increase in the number of eligible gyms, thanks to the extra gyms which were bordering parks and fell inside level 20 s2 cells. There were 24-28 gyms, however, which do not count due to the OSM data being older than I initially anticipated.
 

4.2 How to analyse your own map data

For this to work, you will need a:

• A csv of gyms in your town/suburb
• A copy of osmcoverer

 
The csv needs to be structured with three columns: name, latitude and longitude. Do not include a header row. Example:

Clean and Green Plaque,1.340817,103.743898 Lookout Tower at Yishun Pond,1.425504,103.840164 Autobot Evac,1.254207,103.821722 Clock Tower at Tampines Central Park,1.354006,103.936181 

 
If any of the gym names have commas, either remove those or enclose the gym name in quotation marks. Latitudes and longitudes need to be exact given the precision of level 20 s2 cells – use the Ingress Portal or another source to obtain exact latitudes and longitudes.
 
Go to http://overpass-turbo.eu/s/vs3. Zoom to your town/suburb and click run. This will collect the OSM data. The linked query is backdated to 2016-07-10. Confirmed areas will show up as blue, unconfirmed areas will show up as grey. Export the OSM data; Click “Export” then “Download as GeoJSON”.
 
Open Command Prompt and navigate to the osmcoverer directory. Use the following command:

osmcoverer -markers=gyms.csv input.geojson 

 
osmcoverer will output a file called “markers_within_features.csv” which will contain a list of all gyms inside level 20 s2 cells of OSM parks, etc. It will also generate a .geojson file which has the park polygons, s2 cells and colour-coded markers. This can be visualised by going to http://geojson.io/ and opening the export file. osmcoverer has more options such as being able to generate a level 12 s2 cell overlay for gyms, so it is worth checking out.
 

5.0 Directions for further research

• Furthering restricting the OSM date range: People could check OSM tags of their local EX raid gyms to see if it is possible to further limit the possible date range for OSM data. I would be interested in hearing if you think you have found an EX raid gym with a tag that wasn’t created until after 9 July 2016, or with a tag that was removed before 31 August 2016.

 

• Finding additional OSM tags: People could check OSM tags of their local EX raid gyms to see if anyone can find proof of tags such as natural=heath or leisure=nature_reserve. Berlin players could check gyms within the boundary=physiogeographical mega-nest, which might be able to have EX raids at gyms even when no other OSM tags are present.

 

• Monitoring Niantic’s selection processes over time to check for changes: Niantic paused their non-sponsored EX raid releases in October while they were modifying their system to target parks. Was the missed 1.5 weeks of EX raid invites over DecembeJanuary just a side-effect of Christmas, or has there been another modification? We should definitely keep an eye out for whether Niantic expands to non-park areas again and/or switches to newer OSM data for EX raids.

 
^{And yes, I made a typo in the title :(}