Improved Use of the File System

Table of Contents

Performance

Optimizing Write Speed

It takes a relatively long time to make a single write in the case of save data archives and expanded save data archives, because a lot of processing is carried out for improved robustness and security. Write times may be particularly excessive, depending on the SD Card. The following concepts are recommended to minimize associated adverse effects on performance.

Always keep these concepts in mind during the design phase, whenever writing to a file is necessary. Failure to consider these concepts in the design phase may incur a large cost later if sufficient performance cannot be obtained and revision of software becomes necessary.

For example, consider a situation where 10 items of data of type Data need to be written into the file arc:/data.dat. Assume that the function const Data* GetData(i); gets the pointer to the i-th data item. The following is just pseudo-code. Mounting and error handling are omitted for improved readability. Be sure to carry out proper error handling according to guidelines given in Error Handling, when writing actual code.

First, let's consider the simplest code.

nn::fs::CreateFile("arc:/data.dat", sizeof(Data) * 10);
for (int i = 0; i < 10; ++i)
{
    nn::fs::FileOutputStream f;
    f.Initialize("arc:/data.dat", false);
    f.Seek(sizeof(Data) * i);
    f.Write(GetData(i), sizeof(Data), true);
    f.Finalize();
}
          

In this code, we write to the file once per data item. This process represents a heavy load on the file system. Therefore, consider the following:

nn::fs::CreateFile("arc:/data.dat", sizeof(Data) * 10);
nn::fs::FileOutputStream f;
f.Initialize("arc:/data.dat", false);
for (int i = 0; i < 10; ++i)
{
    f.Write(GetData(i), sizeof(Data), true);
}
f.Finalize();
          

Now the file is only opened once. The load on the system can be expected to decrease proportionally. However, the number of flushes made is still high.

nn::fs::CreateFile("arc:/data.dat", sizeof(Data) * 10);
nn::fs::FileOutputStream f;
f.Initialize("arc:/data.dat", false);
for (int i = 0; i < 10; ++i)
{
    f.Write(GetData(i), sizeof(Data), false);
}
f.Flush();
f.Finalize();
          

If we re-write the code as given above, we can expect the load on the system to be even further reduced by performing only one flush at the end.

However, since sizeof(Data) is relatively small and the overhead required for a single write is large, it may take much longer than expected to write the entire amount. If that happens, we can improve speed by collecting data and writing it all at once, like this:

nn::fs::CreateFile("arc:/data.dat", sizeof(Data) * 10);
nn::fs::FileOutputStream f;
f.Initialize("arc:/data.dat", false);
bit8* buffer = new bit8[sizeof(Data) * 10];
for (int i = 0; i < 10; ++i)
{
    std::memcpy(buffer + sizeof(Data) * i, GetData(i), sizeof(Data));
}
f.Write(buffer, sizeof(Data) * 10, true);
delete [] buffer;
f.Finalize();
          

If you cannot secure a large enough buffer to write all the data at once, you can also gather and write the data in portions, using code like this:

nn::fs::CreateFile("arc:/data.dat", sizeof(Data) * 10);
nn::fs::FileOutputStream f;
f.Initialize("arc:/data.dat", false);
bit8* buffer = new bit8[sizeof(Data) * 5];
for (int j = 0; j < 2; ++j)
{
    for (int i = 0; i < 5; ++i)
    {
        std::memcpy(buffer + sizeof(Data) * i, GetData(5 * j + i), sizeof(Data));
    }
    f.Write(buffer, sizeof(Data) * 5, false);
}
delete [] buffer;
f.Flush();
f.Finalize();
          

As a guideline, consider dividing data up into sizes that can be handled using a buffer size of 500 KB or more. Performance will likely drop significantly if data is written in smaller sizes than this.Finally, Nintendo strongly recommends that you allocate a buffer of 1 MB or larger, when writing excessively large amounts of data (10 MB or more) at once.

Revision History

2012/01/23
Initial version.

CONFIDENTIAL