1<?xml version="1.0" encoding="utf-8"?>
2<html xml:lang="en-US" lang="en-US">
3<head>
4<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
5<meta http-equiv="Content-Style-Type" content="text/css" />
6<link rel="stylesheet" href="../../../css/page.css" type="text/css" />
7<title>Improved Use of the File System</title>
8<style type="text/css">
9<!--
10    strong { color: red; }
11-->
12</style>
13</head>
14<body>
15<h1>Improved Use of the File System</h1>
16
17<h2>Table of Contents</h2>
18    <ul>
19<li><a href="#Performance">Performance</a>
20        <ul>
21<li><a href="#Performance_Write">Optimizing Write Speed</a></li>
22        </ul>
23      </li>
24<li><a href="#History">Revision History</a></li>
25    </ul>
26
27<h2><a name="Performance">Performance</a></h2>
28    <div class="section">
29
30<h3><a name="Performance_Write">Optimizing Write Speed</a></h3>
31      <div class="section">
32        <p>
33It takes a relatively long time to make a single write in the case of save data archives and expanded save data archives, because a lot of processing is carried out for improved robustness and security. Write times may be particularly excessive, depending on the SD Card. The following concepts are recommended to minimize associated adverse effects on performance.
34        </p>
35        <p>
36          <ul>
37<li>Reduce the number of times files are opened and closed.</li>
38<li>Reduce the number of times the cache is flushed.</li>
39<li>Write data all at once using large buffers (500 KB or more).</li>
40          </ul>
41        </p>
42        <p>
43<strong>Always keep these concepts in mind during the design phase, whenever writing to a file is necessary. </strong>Failure to consider these concepts in the design phase may incur a large cost later if sufficient performance cannot be obtained and revision of software becomes necessary.
44        </p>
45        <p>
46For example, consider a situation where 10 items of data of type <CODE>Data</CODE> need to be written into the file <CODE>arc:/data.dat</CODE>. Assume that the function <CODE>const Data* GetData(i);</CODE> gets the pointer to the i-th data item. The following is just pseudo-code. Mounting and error handling are omitted for improved readability. <strong>Be sure to carry out proper error handling according to guidelines given in <a href="./aboutErrorHandling.html"><CODE>Error Handling</CODE></a>, when writing actual code.</strong>
47        </p>
48        <p>
49First, let's consider the simplest code.
50        </p>
51        <p>
52          <code><pre>
53nn::fs::CreateFile("arc:/data.dat", sizeof(Data) * 10);
54for (int i = 0; i &lt; 10; ++i)
55{
56    nn::fs::FileOutputStream f;
57    f.Initialize("arc:/data.dat", false);
58    f.Seek(sizeof(Data) * i);
59    f.Write(GetData(i), sizeof(Data), true);
60    f.Finalize();
61}
62          </pre></code>
63        </p>
64        <p>
65In this code, we write to the file once per data item. This process represents a heavy load on the file system. Therefore, consider the following:
66        </p>
67        <p>
68          <code><pre>
69nn::fs::CreateFile("arc:/data.dat", sizeof(Data) * 10);
70nn::fs::FileOutputStream f;
71f.Initialize("arc:/data.dat", false);
72for (int i = 0; i &lt; 10; ++i)
73{
74    f.Write(GetData(i), sizeof(Data), true);
75}
76f.Finalize();
77          </pre></code>
78        </p>
79        <p>
80 Now the file is only opened once. The load on the system can be expected to decrease proportionally. However, the number of flushes made is still high.
81        </p>
82        <p>
83          <code><pre>
84nn::fs::CreateFile("arc:/data.dat", sizeof(Data) * 10);
85nn::fs::FileOutputStream f;
86f.Initialize("arc:/data.dat", false);
87for (int i = 0; i &lt; 10; ++i)
88{
89    f.Write(GetData(i), sizeof(Data), false);
90}
91f.Flush();
92f.Finalize();
93          </pre></code>
94        </p>
95        <p>
96If we re-write the code as given above, we can expect the load on the system to be even further reduced by performing only one flush at the end.
97        </p>
98        <p>
99However, since sizeof(Data) is relatively small and the overhead required for a single write is large, it may take much longer than expected to write the entire amount. If that happens, we can improve speed by collecting data and writing it all at once, like this:
100        </p>
101        <p>
102          <code><pre>
103nn::fs::CreateFile("arc:/data.dat", sizeof(Data) * 10);
104nn::fs::FileOutputStream f;
105f.Initialize("arc:/data.dat", false);
106bit8* buffer = new bit8[sizeof(Data) * 10];
107for (int i = 0; i &lt; 10; ++i)
108{
109    std::memcpy(buffer + sizeof(Data) * i, GetData(i), sizeof(Data));
110}
111f.Write(buffer, sizeof(Data) * 10, true);
112delete [] buffer;
113f.Finalize();
114          </pre></code>
115        </p>
116        <p>
117 If you cannot secure a large enough buffer to write all the data at once, you can also gather and write the data in portions, using code like this:
118        </p>
119        <p>
120          <code><pre>
121nn::fs::CreateFile("arc:/data.dat", sizeof(Data) * 10);
122nn::fs::FileOutputStream f;
123f.Initialize("arc:/data.dat", false);
124bit8* buffer = new bit8[sizeof(Data) * 5];
125for (int j = 0; j &lt; 2; ++j)
126{
127    for (int i = 0; i &lt; 5; ++i)
128    {
129        std::memcpy(buffer + sizeof(Data) * i, GetData(5 * j + i), sizeof(Data));
130    }
131    f.Write(buffer, sizeof(Data) * 5, false);
132}
133delete [] buffer;
134f.Flush();
135f.Finalize();
136          </pre></code>
137        </p>
138        <p>
139
140        </p>
141        <p>
142<strong>As a guideline, consider dividing data up into sizes that can be handled using a buffer size of 500 KB or more. Performance will likely drop significantly if data is written in smaller sizes than this.</strong>Finally, Nintendo strongly recommends that you allocate a buffer of 1 MB or larger, when writing excessively large amounts of data (10 MB or more) at once.
143        </p>
144      </div>
145
146    </div>
147
148<h2><a name="History">Revision History</a></h2>
149    <div class="section">
150      <dl class="history">
151        <dt>2012/01/23</dt>
152<dd>Initial version.</dd>
153      </dl>
154    </div>
155
156<hr><p>CONFIDENTIAL</p></body>
157</html>