EmbDev.net

Forum: µC & Digital Electronics FAT fs robustness and consistency check


Author: Aldo Dolfi (sysopch)
Posted on:

Rate this post
0 useful
not useful
Hi all,
I am new to the forum.

I am using an embedded FAT filesystem (FATFS 8.3 on SD with Atmel SAM) 
on a datalogger system in which unexpected power failure is a possible 
condition, even if an infrequent event.
I am interested in ways to to increase the robustness of my system as 
much as possible against data and file system corruption as well as 
techniques to check the file system for correctness.

Currently I am thinking of:
- implement two separate partitions:
  -- one for system configuration, mostly read only data, possibly 
written when the unit is attached to a PC. It is very unlikely to have 
an unexpected power failure here (even if a PC crash is always a 
possibility)
  -- one for data logging and variable data. If this one gets corrupted, 
the system should be able to recover, even if with loss of data, 
possibly reformatting

However, I would like to be able to detect any possible file system 
inconsistency as soon as possible and warn the user/take corrective 
actions.

Do you know of any freely available/commercial (open source preferred) 
FAT file system checking routine, suitable for embedded systems?
Repairing features would be nice, but not required.

I was thinking to look at the sources of fsck.vfat but I am sure that 
the problem has been faced and solved before in the embedded world, so I 
thought I would ask first.

Any suggestions on the approch, pointers to docs or code are welcome.
Thanks in advance

Aldo

Author: c-hater (Guest)
Posted on:

Rate this post
0 useful
not useful
Aldo Dolfi wrote:

> I am using an embedded FAT filesystem (FATFS 8.3 on SD with Atmel SAM)
> on a datalogger system in which unexpected power failure is a possible
> condition, even if an infrequent event.
> I am interested in ways to to increase the robustness of my system as
> much as possible against data and file system corruption as well as
> techniques to check the file system for correctness.

FAT isn't designed to handle corruptions. Therefore the only way to get 
a more robust solution is: Avoid corruptions by hardware design.

If a power failure is the source of a possible corruption, the logical 
way to avoid this impact is an early recognition of this condition and a 
secondary power source with enough capacity to allow the completion of 
any pending filesystem transaction. Your task is: Define, what a 
transaction should be. Note: It's not allways an easy task, because it 
may be application dependent!

The simple case is, when filesystem structures must be kept in a 
consistent state only. That's quit easy. Journalling FS works at this 
level (with a different approach).
But such solutions cannot garantuee the inner consistency of any 
particular file. That's completely impossible, because the filesystem 
"driver" has no knowledge of the inner structure of any files except 
those keeping it's own directory informations.

> However, I would like to be able to detect any possible file system
> inconsistency as soon as possible and warn the user/take corrective
> actions.

Bullshit. Avoiding damages is the only non-nonsense solution. If this 
approach accidently fails (what never should happen in a well-done 
design), repair functions can applied to the media outside the embedded 
world, because there is the computing performance and the memory space 
to perform this task in a senseful way and in an acceptable amount of 
time.

Author: Aldo Dolfi (sysopch)
Posted on:

Rate this post
0 useful
not useful
Hi,
thanks for answering

c-hater wrote:
> FAT isn't designed to handle corruptions. Therefore the only way to get
> a more robust solution is: Avoid corruptions by hardware design.

Right, actually we are also considering this approach, I am looking into 
what can be done at software level.
I was referring to file system integrity only and not file integrity, as 
we are already trying to keep application data as consistent as 
possible. This is maybe simple in our case as we can assume that the 
transaction is a write to a single file, so we are making sure we flush 
the file when needed.

>
>> However, I would like to be able to detect any possible file system
>> inconsistency as soon as possible and warn the user/take corrective
>> actions.
>
> Bullshit. Avoiding damages is the only non-nonsense solution. If this
> approach accidently fails (what never should happen in a well-done
> design), repair functions can applied to the media outside the embedded
> world, because there is the computing performance and the memory space
> to perform this task in a senseful way and in an acceptable amount of
> time.

Unfortunately, sh*t happens. So even if I agree that bugs should not be 
there and the file system should not be corrupted, in our case it would 
still be sensible to take some action in case a corruption is detected.
In particular, we do not have an extractable media, nor mass-storage 
emulation, then the PC cannot perform the check/repair operation. In the 
scheme I was sketching above, if I detect a corruption I could tell the 
user: "hey, I have an error, you may not entirely trust your data: 
please back them up and let me clean up".
Clean up could be: reformat the data partition.
But for doing this I need to at least be able to detect the problem.
Loosing the logged data is still preferrable to having the device in an 
unusable state.

Does it make more sense?

Reply

Entering an e-mail address is optional. If you want to receive reply notifications by e-mail, please log in.

Rules — please read before posting

  • Post long source code as attachment, not in the text
  • Posting advertisements is forbidden.

Formatting options

  • [c]C code[/c]
  • [avrasm]AVR assembler code[/avrasm]
  • [code]code in other languages, ASCII drawings[/code]
  • [math]formula (LaTeX syntax)[/math]




Bild automatisch verkleinern, falls nötig
Note: the original post is older than 6 months. Please don't ask any new questions in this thread, but start a new one.