# Forum: µC & Digital Electronics FAT fs robustness and consistency check

 Author: Aldo Dolfi (sysopch) Posted on: 2014-10-24 12:34

Rate this post
 0 ▲ useful ▼ not useful
Hi all,
I am new to the forum.

I am using an embedded FAT filesystem (FATFS 8.3 on SD with Atmel SAM)
on a datalogger system in which unexpected power failure is a possible
condition, even if an infrequent event.
I am interested in ways to to increase the robustness of my system as
much as possible against data and file system corruption as well as
techniques to check the file system for correctness.

Currently I am thinking of:
- implement two separate partitions:
-- one for system configuration, mostly read only data, possibly
written when the unit is attached to a PC. It is very unlikely to have
an unexpected power failure here (even if a PC crash is always a
possibility)
-- one for data logging and variable data. If this one gets corrupted,
the system should be able to recover, even if with loss of data,
possibly reformatting

However, I would like to be able to detect any possible file system
inconsistency as soon as possible and warn the user/take corrective
actions.

Do you know of any freely available/commercial (open source preferred)
FAT file system checking routine, suitable for embedded systems?
Repairing features would be nice, but not required.

I was thinking to look at the sources of fsck.vfat but I am sure that
the problem has been faced and solved before in the embedded world, so I

Any suggestions on the approch, pointers to docs or code are welcome.

Aldo

 Author: c-hater (Guest) Posted on: 2014-10-24 19:17

Rate this post
 0 ▲ useful ▼ not useful
Aldo Dolfi wrote:

> I am using an embedded FAT filesystem (FATFS 8.3 on SD with Atmel SAM)
> on a datalogger system in which unexpected power failure is a possible
> condition, even if an infrequent event.
> I am interested in ways to to increase the robustness of my system as
> much as possible against data and file system corruption as well as
> techniques to check the file system for correctness.

FAT isn't designed to handle corruptions. Therefore the only way to get
a more robust solution is: Avoid corruptions by hardware design.

If a power failure is the source of a possible corruption, the logical
way to avoid this impact is an early recognition of this condition and a
secondary power source with enough capacity to allow the completion of
transaction should be. Note: It's not allways an easy task, because it
may be application dependent!

The simple case is, when filesystem structures must be kept in a
consistent state only. That's quit easy. Journalling FS works at this
level (with a different approach).
But such solutions cannot garantuee the inner consistency of any
particular file. That's completely impossible, because the filesystem
"driver" has no knowledge of the inner structure of any files except
those keeping it's own directory informations.

> However, I would like to be able to detect any possible file system
> inconsistency as soon as possible and warn the user/take corrective
> actions.

Bullshit. Avoiding damages is the only non-nonsense solution. If this
approach accidently fails (what never should happen in a well-done
design), repair functions can applied to the media outside the embedded
world, because there is the computing performance and the memory space
to perform this task in a senseful way and in an acceptable amount of
time.

 Author: Aldo Dolfi (sysopch) Posted on: 2014-10-27 16:56

Rate this post
 0 ▲ useful ▼ not useful
Hi,

c-hater wrote:
> FAT isn't designed to handle corruptions. Therefore the only way to get
> a more robust solution is: Avoid corruptions by hardware design.

Right, actually we are also considering this approach, I am looking into
what can be done at software level.
I was referring to file system integrity only and not file integrity, as
we are already trying to keep application data as consistent as
possible. This is maybe simple in our case as we can assume that the
transaction is a write to a single file, so we are making sure we flush
the file when needed.

>
>> However, I would like to be able to detect any possible file system
>> inconsistency as soon as possible and warn the user/take corrective
>> actions.
>
> Bullshit. Avoiding damages is the only non-nonsense solution. If this
> approach accidently fails (what never should happen in a well-done
> design), repair functions can applied to the media outside the embedded
> world, because there is the computing performance and the memory space
> to perform this task in a senseful way and in an acceptable amount of
> time.

Unfortunately, sh*t happens. So even if I agree that bugs should not be
there and the file system should not be corrupted, in our case it would
still be sensible to take some action in case a corruption is detected.
In particular, we do not have an extractable media, nor mass-storage
emulation, then the PC cannot perform the check/repair operation. In the
scheme I was sketching above, if I detect a corruption I could tell the
user: "hey, I have an error, you may not entirely trust your data:
please back them up and let me clean up".
Clean up could be: reformat the data partition.
But for doing this I need to at least be able to detect the problem.
Loosing the logged data is still preferrable to having the device in an
unusable state.

Does it make more sense?

• $formula (LaTeX syntax)$