blob: ba65d8df75d874e921e9587e674d6a4fa1cc6179 [file] [log] [blame]
Alexander Afanasyevb1314b12012-11-21 18:23:42 -08001
2Obtaining metrics
3=================
4
5To obtain simulation results, you would need to connect to one or more `trace sources <doxygen/group___trace_source_list.html>`_ provided by ndnSIM classes.
6
Alexander Afanasyevf4a03592012-12-10 16:12:34 -08007It is also possible to use existing trace helpers, which collects and aggregates requested statistical information in text files.
8
Alexander Afanasyev59314802012-11-26 14:56:04 -08009.. _trace classes:
Alexander Afanasyevb1314b12012-11-21 18:23:42 -080010
Alexander Afanasyevf4a03592012-12-10 16:12:34 -080011Packet-level trace helpers
12--------------------------
Alexander Afanasyevb1314b12012-11-21 18:23:42 -080013
14- :ndnsim:`ndn::L3AggregateTracer`
15
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -080016 Tracing the aggregate number of Interests/Data packets forwarded by an NDN node
Alexander Afanasyevb1314b12012-11-21 18:23:42 -080017
18 The following example enables tracing on all simulation nodes:
19
20 .. code-block:: c++
21
22 // the following should be put just before calling Simulator::Run in the scenario
23
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -070024 ndn::L3AggregateTracer::InstallAll ("aggregate-trace.txt", Seconds (1.0));
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -080025
Alexander Afanasyevb1314b12012-11-21 18:23:42 -080026 Simulator::Run ();
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -080027
Alexander Afanasyev59314802012-11-26 14:56:04 -080028 ...
Alexander Afanasyevb1314b12012-11-21 18:23:42 -080029
30
31- :ndnsim:`ndn::L3RateTracer`
32
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -080033 Tracing the rate in bytes and in number of packets of Interest/Data packets forwarded by an NDN node
Alexander Afanasyevb1314b12012-11-21 18:23:42 -080034
35 The following example enables tracing on all simulation nodes:
36
37 .. code-block:: c++
38
39 // the following should be put just before calling Simulator::Run in the scenario
40
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -070041 ndn::L3RateTracer::InstallAll ("rate-trace.txt", Seconds (1.0));
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -080042
Alexander Afanasyevb1314b12012-11-21 18:23:42 -080043 Simulator::Run ();
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -080044
Alexander Afanasyev59314802012-11-26 14:56:04 -080045 ...
Alexander Afanasyevb1314b12012-11-21 18:23:42 -080046
Alexander Afanasyev27370c52013-03-12 13:54:08 -070047- :ndnsim:`L2Tracer`
Alexander Afanasyevfc9d9e12013-08-10 14:17:49 -070048
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -070049 This tracer is similar in spirit to :ndnsim:`ndn::L3RateTracer`, but it currently traces only packet drop on layer 2 (e.g.,
50 due to transmission queue overflow).
Alexander Afanasyev27370c52013-03-12 13:54:08 -070051
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -070052 The following example enables tracing on all simulation nodes:
Alexander Afanasyev27370c52013-03-12 13:54:08 -070053
54 .. code-block:: c++
55
Alexander Afanasyev27370c52013-03-12 13:54:08 -070056 // the following should be put just before calling Simulator::Run in the scenario
57
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -070058 L2RateTracer::InstallAll ("drop-trace.txt", Seconds (0.5));
Alexander Afanasyev27370c52013-03-12 13:54:08 -070059
60 Simulator::Run ();
61
62 ...
Alexander Afanasyevb1314b12012-11-21 18:23:42 -080063
64.. note::
65
66 A number of other tracers are available in ``plugins/tracers-broken`` folder, but they do not yet work with the current code.
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -080067 Eventually, we will port most of them to the current code, but it is not our main priority at the moment and would really appreciate help with writing new tracers and porting the old ones.
Alexander Afanasyevb1314b12012-11-21 18:23:42 -080068
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -080069.. _packet trace helper example:
Alexander Afanasyevb1314b12012-11-21 18:23:42 -080070
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -080071Example of packet-level trace helpers
72+++++++++++++++++++++++++++++++++++++
Alexander Afanasyev59314802012-11-26 14:56:04 -080073
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -080074This example (``ndn-tree-tracers.cc``) demonstrates basic usage of :ref:`trace classes`.
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -080075
76In this scenario we will use a tree-like topology, where consumers are installed on leaf nodes and producer is in the root of the tree:
77
Alexander Afanasyevfc9d9e12013-08-10 14:17:49 -070078.. sidebar:: Topology
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -080079
Alexander Afanasyevfc9d9e12013-08-10 14:17:49 -070080 .. aafig::
81 :aspect: 60
82 :scale: 100
83
84 /--------\ /--------\ /--------\ /--------\
85 |"leaf-1"| |"leaf-2"| |"leaf-3"| |"leaf-4"|
86 \--------/ \--------/ \--------/ \--------/
87 ^ ^ ^ ^
88 | | | |
89 \ / \ /
90 \ / \ / 10Mbps / 1ms
91 \ / \ /
92 | | | |
93 v v v v
94 /-------\ /-------\
95 |"rtr-1"| |"rtr-2"|
96 \-------/ \-------/
97 ^ ^
98 | |
99 \ / 10 Mpbs / 1ms
100 +--------\ /--------+
101 | |
102 v v
103 /--------\
104 | "root" |
105 \--------/
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800106
107The corresponding topology file (``topo-tree.txt``):
108
109.. literalinclude:: ../../examples/topologies/topo-tree.txt
110 :language: bash
111 :linenos:
112 :lines: 1-2,27-
113
114Example simulation (``ndn-tree-tracers.cc``) scenario that utilizes trace helpers:
115
116.. literalinclude:: ../../examples/ndn-tree-tracers.cc
117 :language: c++
118 :linenos:
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -0700119 :lines: 21-28,61-
120 :emphasize-lines: 57-58
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800121
122To run this scenario, use the following command::
123
124 ./waf --run=ndn-tree-tracers
125
126The successful run will create ``rate-trace.txt`` and ``aggregate-trace.txt`` files in the current directly, which can be analyzed manually or used as input to some graph/stats packages.
127
Alexander Afanasyevfc9d9e12013-08-10 14:17:49 -0700128.. sidebar:: Graph build using the `R script <http://www.r-project.org/>`_
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800129
Alexander Afanasyevfc9d9e12013-08-10 14:17:49 -0700130 .. image:: _static/root-rates.png
131 :alt: Interest/Data packet rates at the root node
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800132
Alexander Afanasyevfc9d9e12013-08-10 14:17:49 -0700133 .. image:: _static/root-5sec-counts.png
134 :alt: Interest/Data packet counts at the root node in 5-second intervals
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800135
136.. literalinclude:: ../../examples/graphs/rate-graph.R
137 :language: r
138 :linenos:
139
140For more information about R and ggplot2, please refer to `R language manual <http://cran.r-project.org/manuals.html>`_, `ggplot2 module manual <http://docs.ggplot2.org/current/>`_.
Alexander Afanasyevf4a03592012-12-10 16:12:34 -0800141
Alexander Afanasyev27370c52013-03-12 13:54:08 -0700142.. _Example of packet drop tracer (L2Tracer):
143
144Example of packet drop tracer (L2Tracer)
145----------------------------------------
146
Alexander Afanasyevfc9d9e12013-08-10 14:17:49 -0700147.. sidebar:: Topology
148
149 .. image:: _static/topo-tree-25-node.png
150 :alt: 25-node tree topology
151 :width: 550px
152
Alexander Afanasyev27370c52013-03-12 13:54:08 -0700153This example (``ndn-tree-with-l2tracer.cc``) demonstrates basic usage of :ref:`trace classes`.
154
Alexander Afanasyev27370c52013-03-12 13:54:08 -0700155The corresponding topology file (``topo-tree-25-node.txt``):
156
157.. literalinclude:: ../../examples/topologies/topo-tree-25-node.txt
158 :language: bash
159 :linenos:
160 :lines: 2-
161
162Example simulation (``ndn-tree-with-l2tracer.cc``) scenario that utilizes trace helpers:
163
164.. literalinclude:: ../../examples/ndn-tree-with-l2tracer.cc
165 :language: c++
166 :linenos:
167 :lines: 1-
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -0700168 :emphasize-lines: 16,135
Alexander Afanasyev27370c52013-03-12 13:54:08 -0700169
170To run this scenario, use the following command::
171
172 ./waf --run=ndn-tree-with-l2tracer
173
174The successful run will create ``drop-trace.txt`` file in the current directly, which can be analyzed manually or used as input to some graph/stats packages.
175
176For example, the following `R script <http://www.r-project.org/>`_ will build a number of nice graphs:
177
178.. literalinclude:: ../../examples/graphs/drop-graph.R
179 :language: r
180 :linenos:
181
182.. image:: _static/l2-rate-tracer.png
183 :alt: Packet drop rates on routers
184
Alexander Afanasyevf4a03592012-12-10 16:12:34 -0800185.. _cs trace helper:
186
187Content store trace helper
188--------------------------
189
Alexander Afanasyevb1d6b032013-01-18 14:11:11 -0800190- :ndnsim:`ndn::CsTracer`
Alexander Afanasyevf4a03592012-12-10 16:12:34 -0800191
Alexander Afanasyevb1d6b032013-01-18 14:11:11 -0800192 With the use of :ndnsim:`ndn::CsTracer` it is possible to obtain statistics of cache hits/cache misses on simulation nodes.
Alexander Afanasyevf4a03592012-12-10 16:12:34 -0800193
194 The following code enables content store tracing:
195
196 .. code-block:: c++
197
Alexander Afanasyevf4a03592012-12-10 16:12:34 -0800198 // the following should be put just before calling Simulator::Run in the scenario
199
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -0700200 ndn::CsTracer::InstallAll ("cs-trace.txt", Seconds (1));
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -0800201
Alexander Afanasyevf4a03592012-12-10 16:12:34 -0800202 Simulator::Run ();
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -0800203
Alexander Afanasyevf4a03592012-12-10 16:12:34 -0800204 ...
205
206.. - Tracing lifetime of content store entries
207
208.. Evaluate lifetime of the content store entries can be accomplished using modified version of the content stores.
209.. In particular,
210
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800211.. _cs trace helper example:
212
213Example of content store trace helper
214+++++++++++++++++++++++++++++++++++++
215
216This example (``ndn-tree-cs-tracers.cc``) demonstrates basic usage of content store tracer.
217
218In this scenario we will use the same tree-like topology as in :ref:`previous example <packet trace helper example>`, where consumers are installed on leaf nodes and producer is in the root of the tree.
219The main difference is that each client request data from the same namespace: /root/1, /root/2, ... Another small difference is that in this scenario we start our application not at the same time, but 10 ms apart.
220
221Example simulation (``ndn-tree-cs-tracers.cc``) scenario that utilizes trace helpers:
222
223.. literalinclude:: ../../examples/ndn-tree-cs-tracers.cc
224 :language: c++
225 :linenos:
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -0700226 :lines: 21-28,61-
227 :emphasize-lines: 59
Alexander Afanasyevf4a03592012-12-10 16:12:34 -0800228
229
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800230To run this scenario, use the following command::
Alexander Afanasyevf4a03592012-12-10 16:12:34 -0800231
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800232 ./waf --run=ndn-tree-cs-tracers
233
234The successful run will create ``cs-trace.txt``, which similarly to trace file from the :ref:`tracing example <packet trace helper example>` can be analyzed manually or used as input to some graph/stats packages.
235
236
237Application-level trace helper
238------------------------------
239
240- :ndnsim:`ndn::AppDelayTracer`
241
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -0800242 With the use of :ndnsim:`ndn::AppDelayTracer` it is possible to obtain data about for delays between issuing Interest and receiving corresponding Data packet.
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800243
244 The following code enables application-level Interest-Data delay tracing:
245
246 .. code-block:: c++
247
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800248 // the following should be put just before calling Simulator::Run in the scenario
249
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -0700250 ndn::AppDelayTracer::InstallAll ("app-delays-trace.txt");
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -0800251
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800252 Simulator::Run ();
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -0800253
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800254 ...
255
Alexander Afanasyev29dfb982013-01-18 20:04:15 -0800256 Output file format is tab-separated values, with first row specifying names of the columns. Refer to the following table for the description of the columns:
257
258 +-----------------+---------------------------------------------------------------------+
259 | Column | Description |
260 +=================+=====================================================================+
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -0800261 | ``Time`` | simulation time when SeqNo was receivied |
262 +-----------------+---------------------------------------------------------------------+
Alexander Afanasyev29dfb982013-01-18 20:04:15 -0800263 | ``Node`` | node id, global unique |
264 +-----------------+---------------------------------------------------------------------+
265 | ``AppId`` | app id, local unique on the node, not global |
266 +-----------------+---------------------------------------------------------------------+
267 | ``SeqNo`` | seq number of the Interest-Data |
268 +-----------------+---------------------------------------------------------------------+
269 | ``Type`` | Type of delay: |
270 | | |
271 | | - ``LastDelay`` means that ``DelayS`` and ``DelayUS`` represent |
272 | | delay between last Interest sent and Data packet received |
273 | | - ``FullDelay`` means that ``DelayS`` and ``DelayUS`` represent |
274 | | delay between first Interest sent and Data packet received |
275 | | (i.e., includes time of Interest retransmissions) |
276 +-----------------+---------------------------------------------------------------------+
277 | ``DelayS`` | delay value, specified in seconds |
278 +-----------------+---------------------------------------------------------------------+
279 | ``DelayUS`` | delay value, specified in microseconds (10^-6) |
280 +-----------------+---------------------------------------------------------------------+
Alexander Afanasyev400aae12013-01-19 13:27:52 -0800281 | ``RetxCount`` | number of Interest retransmissions (for LastDelay always equal to 1)|
282 +-----------------+---------------------------------------------------------------------+
Alexander Afanasyev1a0fff62013-01-19 14:29:51 -0800283 | ``HopCount`` | combined number of number of hops for Interest and Data packet. |
284 | | Note that HopCount is increased anytime somebody calls Send method |
285 | | on a face, including delivery of Interest/Data to application via |
286 | | an AppFace (but not from application to ndn::L3Protocol!). |
287 | | |
288 | | One consequence is that Interests satisfied by an app will have |
289 | | even hop count (min hop count = 2), and Interests satisfied from |
290 | | caches will have off hop count (min hop count = 1) |
291 +-----------------+---------------------------------------------------------------------+
Alexander Afanasyev29dfb982013-01-18 20:04:15 -0800292
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800293.. _app delay trace helper example:
294
295Example of application-level trace helper
296+++++++++++++++++++++++++++++++++++++++++
297
298This example (``ndn-tree-app-delay-tracer.cc``) demonstrates basic usage of application-level Interest-Data delay tracer.
299
Alexander Afanasyevfc8425c2013-01-31 13:33:49 -0800300In this scenario we will use the same tree-like topology as in :ref:`packet trace helper example <packet trace helper example>`, where consumers are installed on leaf nodes and producer is in the root of the tree and clients request data from the same namespace: /root/1, /root/2, ...
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800301
302Example simulation (``ndn-tree-app-delay-tracer.cc``) scenario that utilizes trace helpers:
303
304.. literalinclude:: ../../examples/ndn-tree-app-delay-tracer.cc
305 :language: c++
306 :linenos:
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -0700307 :lines: 21-28,61-
308 :emphasize-lines: 59
Alexander Afanasyevdb64ff12013-01-18 16:37:31 -0800309
310
311To run this scenario, use the following command::
312
313 ./waf --run=ndn-tree-app-delay-tracer
314
315The successful run will create ``app-delays-trace.txt``, which similarly to trace file from the :ref:`packet trace helper example <packet trace helper example>` can be analyzed manually or used as input to some graph/stats packages.
316
Alexander Afanasyevf4a03592012-12-10 16:12:34 -0800317
Alexander Afanasyevb5e77d82013-04-10 15:55:26 -0700318Other types of stats
319--------------------
320
321.. _periodic tracing of Pending Interest Table (PIT) size:
322
323Periodic tracing of Pending Interest Table (PIT) size
324+++++++++++++++++++++++++++++++++++++++++++++++++++++
325
326This example (``ndn-simple-with-pit-count-stats.cc``) shows how you can periodically print out current size of PIT on the selected nodes.
327
328.. literalinclude:: ../../examples/ndn-simple-with-pit-count-stats.cc
329 :language: c++
330 :linenos:
331 :lines: 20-26,47-
Alexander Afanasyev3fe94dc2013-08-09 17:12:12 -0700332 :emphasize-lines: 9-20,43-46,59-61
Alexander Afanasyevb5e77d82013-04-10 15:55:26 -0700333
334To run this scenario, use the following command::
335
336 ./waf --run=ndn-simple-with-pit-count-stats
337