We use QField for field data collection and QGIS for office processing. Our workflow:
- Field workers collect data in “sketch” layers via QField, synced to QField Cloud.
- Office workers verify this data in QGIS and transfer it to the main database.
- Verified data is deleted from the sketch layer.
- Updated database (read-only) and condensed sketch layer are synced back to the cloud.
Questions:
-
If field workers upload new data while office processing is ongoing, and then office workers upload their processed data, will the field workers’ new data be overwritten?
-
How can we prevent data loss?
- If we change field workers’ roles to “Reader” during office work:
a) Can they still collect data locally?
b) What happens when they try to upload this data later?
c) Will local changes persist until they regain “Editor” status?
Any insights or better ideas how to manage data input from QField and QGIS would be appreciated.
Imported from GitHub discussion by @meyerlor on 2024-09-12T08:21:31Z
Hi meyerlor,
Your first question: Yes, the data can be overwritten, but only if the project has the option setting to “Overwrite conflicts” activated. If deactivated, the manager will have the option to decide which changes to apply or reject.
https://docs.qfield.org/reference/qfieldcloud/conflicts/
Regarding your second question, option a, if the user is connected to the internet, their role will change and they won’t be able to create changes. Point b, it is likely that the changes can’t be pushed. As for point c, yes, the changes will remain on the device, and if it’s not possible to push the changes, you could try to retrieve your data from the mobile device by clicking Settings (Gear icon) > Open Project Folder > data.gpkg, then use the three dots on the right side and the function “Send to…” in order to send your data to your computer.
As for some tips, when updating existing features based on field conditions, plan and designate the features each user will update.
Imported from GitHub comment by @SeqLaz on 2024-09-12T16:51:55Z
Thanks for your answer!
Regarding conflict management:
When a field worker uploads data and after that an office worker saves changes and then tries to upload the geopackage, the office geopackage will have a newer timestamp - will QFieldSync/QfieldCloud actually recognize that there is a conflict?
Is this manual conflict managment option also available on a self hosted QFieldCloud instance?
Imported from GitHub comment by @meyerlor on 2024-09-13T06:26:05Z
When new changes are uploaded from a computer to the cloud, these changes take priority and overwrite the existing files. This means that the newly uploaded data will be the version that syncs across all devices. To prevent data loss and manage conflicts effectively, there is a workflow:
- Field devices should first upload their changes to QField (and make sure that all the data was correctly pushed).
- Then, download the latest data onto the QGIS and begin making updates.
- After updating, push the changes to the QFieldCloud.
- And before heading out for fieldwork, synchronize QField with the most recent data.
This helps ensure that all changes are preserved and conflicts are minimized.
The same applies to manual conflict resolution in a self-hosted QFieldCloud.
Imported from GitHub comment by @SeqLaz on 2024-09-13T07:10:29Z
OK, thanks for clearifying and your input! I just saw, that the conflict-management button is also coming to self-hosted instances .
Your proposed workflow is fine, but especially in a larger environment, field work and office work needs to be done simultanously - so i guess i have to opt for PostGres, there changes always happen feature-wise and the danger to overwrite a work-day of a field worker who forgot to upload data is less likely i think! (Correct me if i’m wrong).
Imported from GitHub comment by @meyerlor on 2024-09-17T10:38:57Z
Yes the option for overwrite conflict will be available soon, and you are completely right; opting for a PostgreSQL (PostGIS) database is a better option.
Imported from GitHub comment by @SeqLaz on 2024-09-17T16:35:38Z
Oh I enjoy this conversation, educative!
Like Meyelor, I also have a situation I’ve been to solve.
I work in a Fibre Telecom company as the GIS guy. I’m to build their spatial database from ground up, so I QField for field survey.
Here are my encounters:
I have couple of layers ‘Handhole, Splitter Box, Cabinet, Closure etc’.
- Each layer has it own Tag ID(i.e for Handhole: CV-HH-R1-0001), how can I autogenerate this ID serially, that is after point 0001 is done next point gets CV-HH-R1-0002
- I wish to have a table where if either a Splitter Box or Closure is in a Handhole, it keeps the record store in individual layer. For instance, I checked the box the a Closure in within a Handhole, the Tag ID and other records surveyed also goes to the Closure layer.
Thank you
Imported from GitHub comment by @Asuduq on 2024-09-19T16:36:25Z
Hi Asuduq ,
I highly recommend opening a new discussion for this topics.
For the “fid” field of the layers used in surveys, I suggest using the expression epoch(now()) for the “fid” field.
Regarding the creation of a serial incremental ID on strings, you could use an expression similar to the following, but I highly recommend using a UUID for relations too.
You have two options for creating UUID values:
- You can use the “UUID Generator” widget, which will automatically generate unique UUID values for each entry.
- Alternatively, you can use the “Text Edit” widget and manually generate UUID values using the
uuid() expression.
'CV-HH-R1-' ||
CASE
WHEN (count("fid") + 1) < 10 THEN concat('000', (count("fid") + 1))
WHEN (count("fid") + 1) < 100 THEN concat('00', (count("fid") + 1))
WHEN (count("fid") + 1) < 1000 THEN concat('0', (count("fid") + 1))
WHEN (count("fid") + 1) < 10000 THEN concat('', (count("fid") + 1))
END
And regarding the option to store records, it is to create “Relations” within those layers. This way, you can choose which feature has a relation with other features.
Imported from GitHub comment by @SeqLaz on 2024-09-19T17:43:52Z
