Is there a way to physically access the stored data of a QFieldCloud project on the device?
One of my users is experiencing data loss when he syncs his tablet after field work, and I wanted to do a backup before he does next time to try and isolate the problem… but I can’t seem to find the project on my device!
Imported from GitHub discussion by @OBVYamaska on 2022-05-06T21:25:01Z
suricactus , nirvn , I’m trying to get your attention on this matter, which seems like a very high priority: we confirmed that we experience some data loss during QFieldCloud sync process.
The recurring behavior is that only data taken at the end of the day is lost, for some reason.
The suspected reasons might be linked to the hour itself (I failed to see how this could pose a problem), or rather to the volume of data that has to sync (thus, working for the first observations, but failing at some point at the end of a work day).
This poses real issues as of if we will continue to use the app this summer, so I would really like to investigate this with you.
As I can’t find a way to manually extract QFieldSync data before a sync, I can’t diagnose the problem on my end… any help?
Imported from GitHub comment by @OBVYamaska on 2022-05-16T16:57:49Z
Hey, OBVYamaska , sorry for the late reply, but I am busy with other stuff too. Thank you for checking QFieldCloud BETA. If you really want to bring our attention, we also do provide support, please drop us an email at info [at] opengis.ch .
On your questions.
Are you still seing the lost data in the “Changes” page on the web interface? If yes, then we are all good. I think would be a good move to allow users to reapply changes, no matter their status.
On the access to the file system, please check the latest mater builds of QField, yesterday we merged a new interface to access your files, including for cloud projects. More info here: QML project/dataset picker by nirvn · Pull Request #2912 · opengisch/QField · GitHub
Imported from GitHub comment by @suricactus on 2022-05-20T02:36:56Z
Hi suricactus , no problem, I totally understand. I also thought this was the best way to get help, but I’ll write an email next time, sorry!
So, I do see a lot of data tagged as “error” in the “Changes” page on the web interface, but it’s hard for me to tell is some are the missing datas or not. I’ll check with my coworker to try to pinpoint some of the missing ones. You tell me I simply have to apply them and they should follow?
I’ll also check the latest build. I get I have to install it manually, as I don’t see an update on the PlayStore?
Thanks again!
Imported from GitHub comment by @OBVYamaska on 2022-05-24T14:27:19Z
I tried to commit the changes that were blocked (which seemed like a lot), but now they are marked as pending (since this morning, so several hours), and the project won’t sync on the tablets anymore…
Imported from GitHub comment by @OBVYamaska on 2022-05-24T20:50:08Z
Hi suricactus ,
So I tried various things:
I totally get that the team is already busy. But I believe data loss should be a high priority. I’m a true advocate of your work and I’m pushing a lot of partners to switch from ESRI to QGIS/QField, but this issue as pushed an already tight project to a late project with half the data inaccessible… I would be happy to at least help resolve the issue for others! 
Again, thanks a lot for your help.
Imported from GitHub comment by @OBVYamaska on 2022-06-02T16:09:41Z
OBVYamaska , you most definitely can access cloud project data stored on your device using the new interface. Simply open that locally mirrored cloud project on your device and select “open project folder” from the main menu. You can then send out datasets via email/messenger or export them into public parts of your phone storage.
Imported from GitHub comment by @nirvn on 2022-06-02T23:46:10Z
nirvn , that’s really good new!
That being said, I installed the 2912 version, and while I see the option to export cloud (and other) projects, I don’t have access to projects that were downloaded locally on other versions of QField.
To clarify, I have a project started under v.2.1.4 Bumblebee and QFieldCloud. At some point, I started to lose data during the syncs, and now, the project will no longer sync, but I still have data inside.
I can’t see this project in the 2912 installation. When I connect to my account, the project is stated as unavailable locally.
What am I doing wrong?
Imported from GitHub comment by @OBVYamaska on 2022-06-03T05:38:45Z
OBVYamaska , ah, you will have to wait for the next version of QField on the play store (2.2, in last week of June). If you are installing a PR APK, the internal folder where cloud projects are stored isn’t the same as QField served on play store. So hold on a few more days for now
Imported from GitHub comment by @nirvn on 2022-06-03T07:07:40Z
Awesome, I’ll look forward to that
This will solve the problem of the datas stuck on the device.
Any clue on how to deal with the ones stuck on the cloud with an error, and won’t sync to my computer?
Imported from GitHub comment by @OBVYamaska on 2022-06-03T15:44:51Z
OBVYamaska , the time as come sir you can download the QField 2.2.0 APK here (Release 2.2.0 - Coordinated Capybara · opengisch/QField · GitHub) or subscribe to the tester channel on the play store. Once installed, you can access your data by using the ‘Open Project Folder’ action in the main menu:
That’ll lead you to a folder view with project file(s) and dataset(s) displayed:
Use the 3-dot menu against the dataset(s) you want to access the ‘Send to…’ action, et voila.
Imported from GitHub comment by @nirvn on 2022-07-07T10:45:35Z
I saw it yesterday, and I’m really excited to try it!
So this way, I should be able to recover the datas that are stuck on the device. Awesome
For my understanding, will this also give me access to the datas that were synced from this device, but are no longer visible (those that are marked with an error on the cloud server)? Are they still on the device, but not shown on the map, or are they only on the server? And if this is the later, is there a way to recover them?
I’ll have my hands on the tablet tomorrow and try this all.
Again, thanks for your great work!
Imported from GitHub comment by @OBVYamaska on 2022-07-07T14:35:35Z
OBVYamaska , no, that wouldn’t be stored in your phone anymore. However, you most likely can retrieve individual data push that have error’ed out by looking at the raw JSON of specific changes:
That doesn’t scale up that well when it comes to recovering a larger number of features. I think you’re in touch with Linda, she can expand on support we can offer there
Imported from GitHub comment by @nirvn on 2022-07-08T02:40:46Z
Hey Mathieu,
I think the Raw JSON will be the way to go this time. I will be able to reconstruct my data this way.
I’m taking a chance here, but could it be possible to get a bulk export of the changes from this project?
I have to sort through hundreds of changes, so having everything in a text form instead of the online tool would accelerate everything a lot 
If it’s too much trouble, I’ll simply do it one by one.
Thanks again!
Notez que compte-tenu de la situation actuelle, je suis en télétravail. Veuillez me rejoindre prioritairement par courriel. Si une conversation téléphonique est nécessaire, veuillez m’en informer pour que nous prenions des arrangements conséquents.
Michel Landry, B.Sc. Géomatique, M. Env.
Gestionnaire de projets
[Logo signature courriel]http://www.obv-yamaska.qc.ca/
160, rue Cowie, bureau 201
Granby (Québec) J2G 3V3
450-956-1164, poste 224
@.@.>
www.obv-yamaska.qc.cahttp://www.obv-yamaska.qc.ca/
From: Mathieu Pellerin @.>
Sent: July 7, 2022 22:41 PM
To: opengisch/QField @.>
Cc: Michel Landry @.>; Mention @.>
Subject: Re: [opengisch/QField] Data loss while QFieldCloud sync; Physical access to QFieldCloud data on device (Discussion #2867)
OBVYamaskahttps://github.com/OBVYamaska , no, that wouldn’t be stored in your phone anymore. However, you most likely can retrieve individual data push that have error’ed out by looking at the raw JSON of specific changes:
[image]https://user-images.githubusercontent.com/1728657/177905844-91df7853-ac22-4750-96b1-57962430d886.png
That doesn’t scale up that well when it comes to recovering a larger number of features. I think you’re in touch with Linda, she can expand on support we can offer there
—
Reply to this email directly, view it on GitHubhttps://github.com/opengisch/QField/discussions/2867#discussioncomment-3104427, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AYIJZEIHKAX7QYAIIT2M643VS6ILRANCNFSM5VJJGADQ.
You are receiving this because you were mentioned.Message ID: @.@.>>
Imported from GitHub comment by @OBVYamaska on 2022-07-13T01:51:48Z
Oh, and also… any idea how my project became corrupted in the first place?
So I can avoid this in the future 
Thanks!
Notez que compte-tenu de la situation actuelle, je suis en télétravail. Veuillez me rejoindre prioritairement par courriel. Si une conversation téléphonique est nécessaire, veuillez m’en informer pour que nous prenions des arrangements conséquents.
Michel Landry, B.Sc. Géomatique, M. Env.
Gestionnaire de projets
[Logo signature courriel]http://www.obv-yamaska.qc.ca/
160, rue Cowie, bureau 201
Granby (Québec) J2G 3V3
450-956-1164, poste 224
@.@.>
www.obv-yamaska.qc.cahttp://www.obv-yamaska.qc.ca/
From: Mathieu Pellerin @.>
Sent: July 7, 2022 22:41 PM
To: opengisch/QField @.>
Cc: Michel Landry @.>; Mention @.>
Subject: Re: [opengisch/QField] Data loss while QFieldCloud sync; Physical access to QFieldCloud data on device (Discussion #2867)
OBVYamaskahttps://github.com/OBVYamaska , no, that wouldn’t be stored in your phone anymore. However, you most likely can retrieve individual data push that have error’ed out by looking at the raw JSON of specific changes:
[image]https://user-images.githubusercontent.com/1728657/177905844-91df7853-ac22-4750-96b1-57962430d886.png
That doesn’t scale up that well when it comes to recovering a larger number of features. I think you’re in touch with Linda, she can expand on support we can offer there
—
Reply to this email directly, view it on GitHubhttps://github.com/opengisch/QField/discussions/2867#discussioncomment-3104427, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AYIJZEIHKAX7QYAIIT2M643VS6ILRANCNFSM5VJJGADQ.
You are receiving this because you were mentioned.Message ID: @.@.>>
Imported from GitHub comment by @OBVYamaska on 2022-07-13T01:53:33Z
