blob: b72ba7cdcb31bcf883b7108f851d02dd3376d1b6 [file] [log] [blame] [view]
Vince Lehmance7f4e92015-07-14 15:22:07 -05001Experiments
2===========
3
4Mini-NDN includes an experimentation framework which allows a user to create and automate
5networking experiments. Users can run existing experiments, included with Mini-NDN, or define
6their own custom experiment.
7
8## Running existing experiments
9
10Mini-NDN includes three example experiments that can be used to test the network or as reference
Ashlesh Gawandee626b632016-08-12 16:50:14 -050011for custom experiment implementations. `ndn-tools` must be installed to run the example
12experiments as each experiment uses both `ndnpingserver` and `ndnping`. Please see
13[INSTALL.md](../INSTALL.md) for instructions on installing `ndn-tools`.
Vince Lehmance7f4e92015-07-14 15:22:07 -050014
Ashlesh Gawandee626b632016-08-12 16:50:14 -050015To see a list of the available experiments, run Mini-NDN using the `--list-experiments` parameter:
Vince Lehmance7f4e92015-07-14 15:22:07 -050016
17 minindn --list-experiments
18
Ashlesh Gawandee626b632016-08-12 16:50:14 -050019To run an experiment, provide the experiment name as an argument to the `--experiment` parameter:
Vince Lehmance7f4e92015-07-14 15:22:07 -050020
21 sudo minindn --experiment=pingall
22
23Each experiment will run until completion or exit if there is an error setting up the
24test environment.
25
26The three included experiments are set up using the same starting
27configuration. Each node runs NFD, NLSR, and an ndnpingserver which advertises the node's
28site name. After a waiting period to allow the network to converge (default is 60 seconds),
29the convergence status of the network is checked. If each node's FIB does not have an entry
30for every other node's router name and advertised prefix, the experiment is aborted and an error
31is reported.
32
33#### Common experiment parameters
34
Ashlesh Gawandee626b632016-08-12 16:50:14 -050035The time allowed for convergence (in seconds) can be configured using the `--ctime` parameter:
Vince Lehmance7f4e92015-07-14 15:22:07 -050036
37 sudo minindn --ctime=30 ...
38
39After the experiment has finished running, the command-line interface (CLI) will be launched and the
40user can then interact with the test environment. To disable the CLI and instead exit Mini-NDN
Ashlesh Gawandee626b632016-08-12 16:50:14 -050041as soon as the experiment has finished, use the `--no-cli` parameter:
Vince Lehmance7f4e92015-07-14 15:22:07 -050042
43 sudo minindn --no-cli ...
44
Ashlesh Gawandee626b632016-08-12 16:50:14 -050045To ping only a percentage of nodes `--pct-traffic` can be set.
46
47 sudo minindn --pct-traffic=0.5 ...
48
49The above command will ping only 50% of other nodes from each node.
50The default value is 1 i.e. ping every other node.
51
52To move the experiment results to a results directory from the working directory
53after the experiment is complete (either --no-cli or quit) the following option
54can be used:
55
56 sudo minindn --result-dir /home/mydir/result-dir ...
57
Vince Lehmance7f4e92015-07-14 15:22:07 -050058The included experiments are described in detail below along with additional
59parameters that can be provided to modify the execution of the experiments.
60
61### Pingall experiment
62
63**Scenario**: Each node in the network simultaneously pings every other node in the network at a
64one second interval.
65
Ashlesh Gawandee626b632016-08-12 16:50:14 -050066**Experiment ID**: `--pingall`
Vince Lehmance7f4e92015-07-14 15:22:07 -050067
Ashlesh Gawandee626b632016-08-12 16:50:14 -050068The number of pings sent in the experiment can be configured using the `--nPings` parameter:
Vince Lehmance7f4e92015-07-14 15:22:07 -050069
70 sudo minindn --experiment=pingall --nPings=120
71
Ashlesh Gawandee626b632016-08-12 16:50:14 -050072By default, `--nPings` is 300 for the Pingall experiment.
Vince Lehmance7f4e92015-07-14 15:22:07 -050073
74### Failure experiment
75
76**Scenario**: Each node in the network simultaneously pings every other node in the network at a
77one second interval. After 60 seconds, the node with the name "csu" is brought down. The node is
78left in a failed state for 120 seconds while the other nodes continue pinging. After this period,
79the failed node is recovered and pings are collected for an additional 90 seconds.
80
Ashlesh Gawandee626b632016-08-12 16:50:14 -050081**Experiment ID**: `--failure`
Vince Lehmance7f4e92015-07-14 15:22:07 -050082
Ashlesh Gawandee626b632016-08-12 16:50:14 -050083`--nPings` is 300 for the Failure experiment and cannot be modified.
Vince Lehmance7f4e92015-07-14 15:22:07 -050084
85### Multiple failure experiment
86
87**Scenario**: Each node in the network simultaneously pings every other node in the network at a
88one second interval. After 60 seconds, the first node in the network will be brought down and remain
89in failed state for 60 seconds. After the failure period, the node is recovered and the network
90is allowed to recover for 60 seconds. After the recovery period, the next node will go through this
91failure and recovery process. Once every node in the network has gone through the failure and
92recovery process, the experiment will end.
93
Ashlesh Gawandee626b632016-08-12 16:50:14 -050094**Experiment ID**: `--multiple-failure`
Vince Lehmance7f4e92015-07-14 15:22:07 -050095
Ashlesh Gawandee626b632016-08-12 16:50:14 -050096`--nPings` is dependent on the size of the topology being tested. 120 pings are scheduled for
Vince Lehmance7f4e92015-07-14 15:22:07 -050097each node's failure/recovery period as well as an additional 60 pings for the initial collection
98period.
99
Ashlesh Gawandee626b632016-08-12 16:50:14 -0500100### MCN failure experiment
101
102**Scenario**: This is exactly like the failure experiment but instead of failing the node named "csu" it fails the most connected node (MCN) i.e the node with the most links.
103
104Experiment ID: `--failure-mcn`
105
106### Experiment data
107
dmcoomes80eeea12017-10-27 12:49:10 -0500108The ping data is stored at `/tmp/minindn/node-name/ping-data`.
Ashlesh Gawandee626b632016-08-12 16:50:14 -0500109
dmcoomes80eeea12017-10-27 12:49:10 -0500110The ping server log is stored at `/tmp/minindn/node-name/ping-server`
Ashlesh Gawandee626b632016-08-12 16:50:14 -0500111
Vince Lehmance7f4e92015-07-14 15:22:07 -0500112## Creating custom experiments
113
114Mini-NDN provides a simple Python based framework which allows a user to define their own experiment
115and run it from the command line.
116
117To create an experiment, follow these steps:
118
Ashlesh Gawandee626b632016-08-12 16:50:14 -05001191. Create a Python source file for the experiment in the `ndn/experiments` directory.
Vince Lehmance7f4e92015-07-14 15:22:07 -0500120
Ashlesh Gawandee626b632016-08-12 16:50:14 -0500121 e.g.) `ndn/experiments/example.py`
Vince Lehmance7f4e92015-07-14 15:22:07 -0500122
Ashlesh Gawandee626b632016-08-12 16:50:14 -05001232. Derive the experiment from the `Experiment` base class defined in
124 `ndn/experiments/experiment.py`.
Vince Lehmance7f4e92015-07-14 15:22:07 -0500125
126 #!/usr/bin/python
127
128 from ndn.experiments.experiment import Experiment
129
130 class ExampleExperiment(Experiment):
131 def __init__(self, args):
132 Experiment.__init__(self, args)
133
Ashlesh Gawandee626b632016-08-12 16:50:14 -05001343. Override the `setup()` method to define how the experiment should be initialized
Vince Lehmance7f4e92015-07-14 15:22:07 -0500135
136 e.g.) Run an ndnping server in the background on each node
137
138 def setup(self):
139 for host in self.net.hosts:
140 host.cmd("ndnpingserver host.name &")
141
142
Ashlesh Gawandee626b632016-08-12 16:50:14 -05001434. Override the `run()` method to define how the experiment should behave
Vince Lehmance7f4e92015-07-14 15:22:07 -0500144
145 e.g.) Obtain the NFD status of each node and save it to file
146
147 def run(self):
148 for host in self.net.hosts:
Ashlesh Gawandef932a182016-12-19 23:45:26 -0600149 host.cmd("nfdc status report > status.txt")
Vince Lehmance7f4e92015-07-14 15:22:07 -0500150
Ashlesh Gawandee626b632016-08-12 16:50:14 -05001515. Register the experiment with the `ExperimentManager` to make the experiment runnable from the
Vince Lehmance7f4e92015-07-14 15:22:07 -0500152command line.
153
154 Experiment.register("example-name", ExampleExperiment)
155
156The experiment can then be run from the command-line using the name registered.
157"example-name" in the above example:
158
159 sudo minindn --experiment=example-name
160
161### Full example experiment code
162
163 #!/usr/bin/python
164
165 from ndn.experiments.experiment import Experiment
166
167 class ExampleExperiment(Experiment):
168 def __init__(self, args):
169 Experiment.__init__(self, args)
170
171 def setup(self):
172 for host in self.net.hosts:
173 host.cmd("ndnpingserver host.name &")
174
175 def run(self):
176 for host in self.net.hosts:
Ashlesh Gawandee626b632016-08-12 16:50:14 -0500177 # By default status.txt would be stored
dmcoomes80eeea12017-10-27 12:49:10 -0500178 # at /tmp/minindn/host/status.txt
Ashlesh Gawandef932a182016-12-19 23:45:26 -0600179 host.cmd("nfdc status report > status.txt")
Vince Lehmance7f4e92015-07-14 15:22:07 -0500180
181 Experiment.register("example-name", ExampleExperiment)
Ashlesh Gawande501d4d62017-10-25 13:12:11 -0500182
183## Passing arbitrary arguments to experiments
184
185One can pass any arbitrary argument to the Mini-NDN command line
186as long as the arguments don't clash with Mini-NDN's arguments.
187This feature allows users to pass any argument to Mini-NDN and process
188them in an experiment without modifying Mini-NDN's core.
189
190Please look at `ndn/experiments/arbitrary_arguments_experiment.py`
191to see how these arguments can be accessed. To have the experiment
192options printed in `sudo minindn --list-experiments` when using
193arbitrary arguments one can add the static `arguments` method as
194shown in the aforementioned experiment.
195
196To run the experiment:
197
198 sudo minindn --experiment arbitrary-arguments --ds 400 --logging false
199
200The experiment will print out the supplied arbitrary values which are --ds
201and --logging. --experiment is a fixed argument of Mini-NDN.