Working with larger queries

While the DDb allows access to all plant records that the BSBI holds, there are some limits on the number of results that can be viewed or downloaded.  Up to 50,000 can be viewed in the on-line results table. 

Users without extra admin permissions can download or have e-mailed to them, up to 150,000 records in one go.  If you need to exceed these limits then please ask for assistance by email to tom.humphrey@bsbi.org.  The only likely scenario for this would be a VCR wanting to work on a whole county data set, eg for a Flora, Checklist or Rare Plant Register.  The resultant very large files can be supplied via a file sharing website, eg. Dropbox or similar.

Guidance for successful searches

The following guidelines to help ensure that large searches work reliably (rather than failing due to web-browser time-outs or lack of memory) and that they don't negatively affect the running of the system.

• Consider if you can weed out data that you don't need before searching, e.g. you may need only recent records (rather than those dating back to 1700) or you may be able to exclude a large chunk of data that you already know about (e.g. records already in MapMate, or from a recent flora data set that you already have access to).
• Use the 'Email search results' option, rather than downloading the results directly. This avoids problems with web browser time-outs and lets the server run your search task more efficiently. You can email results straight away without having to run the query first in the web browser (where the results will have been capped at 50,000 anyway).
• If possible download large result sets as a csv (text) file rather than in Excel or Ods format. Delimited-text can still be easily loaded into a spreadsheet, but is a far simpler and quicker format to generate on the server.
• Please avoid rerunning the same query multiple times. If you are unsure if your query has worked then, before restarting it, check in your search history, to see if your search is still running or if there is a previous result set waiting that you can retrieve quickly. If a slow query fails more than once then please seek assistance rather than retrying it - having failed once it will probably fail every time.
• Consider whether your query can sensibly be broken up into smaller parts - but only within reason: splitting a slow query into a few parts (e.g. partitioning by county or date-class) is probably quick and worthwhile, whereas having to run it as many separate searches is a waste of time - in that case please seek advice about how best to run the search in one go).
• If you don't need individual records, only a summary then it's usually quicker to search for the grouped summary directly rather than downloading a large record set and analysing it off-line in a spreadsheet.

The first bullet point, above, is from a practical viewpoint perhaps the most important.  In many circumstances, it is much more useful to just view or download a subset of records, or a summary of them.  It is also quicker.  Searching for records of Adoxa moschatellina returns (as of September 2023) 42,976 records, the query taking 40 to 55 seconds.  That is a bit of a wait, and do you really want to wade through that many records to find the ones of interest to you?  Probably not. 

Restricting the search to a single vice-county (vc94) returns (as of September 2023) 220 records, and the query runs in about 2 seconds.  Restricting the query to vc94 and to records since 2000, currently returns 134 records, and the query runs in less than one second; so both much faster and easier to find the records of particular interest to you.