Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 1 | Experiments |
| 2 | =========== |
| 3 | |
| 4 | Mini-NDN includes an experimentation framework which allows a user to create and automate |
| 5 | networking experiments. Users can run existing experiments, included with Mini-NDN, or define |
| 6 | their own custom experiment. |
| 7 | |
| 8 | ## Running existing experiments |
| 9 | |
| 10 | Mini-NDN includes three example experiments that can be used to test the network or as reference |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 11 | for custom experiment implementations. `ndn-tools` must be installed to run the example |
| 12 | experiments as each experiment uses both `ndnpingserver` and `ndnping`. Please see |
| 13 | [INSTALL.md](../INSTALL.md) for instructions on installing `ndn-tools`. |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 14 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 15 | To see a list of the available experiments, run Mini-NDN using the `--list-experiments` parameter: |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 16 | |
| 17 | minindn --list-experiments |
| 18 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 19 | To run an experiment, provide the experiment name as an argument to the `--experiment` parameter: |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 20 | |
| 21 | sudo minindn --experiment=pingall |
| 22 | |
| 23 | Each experiment will run until completion or exit if there is an error setting up the |
| 24 | test environment. |
| 25 | |
| 26 | The three included experiments are set up using the same starting |
| 27 | configuration. Each node runs NFD, NLSR, and an ndnpingserver which advertises the node's |
| 28 | site name. After a waiting period to allow the network to converge (default is 60 seconds), |
| 29 | the convergence status of the network is checked. If each node's FIB does not have an entry |
| 30 | for every other node's router name and advertised prefix, the experiment is aborted and an error |
| 31 | is reported. |
| 32 | |
| 33 | #### Common experiment parameters |
| 34 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 35 | The time allowed for convergence (in seconds) can be configured using the `--ctime` parameter: |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 36 | |
| 37 | sudo minindn --ctime=30 ... |
| 38 | |
| 39 | After the experiment has finished running, the command-line interface (CLI) will be launched and the |
| 40 | user can then interact with the test environment. To disable the CLI and instead exit Mini-NDN |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 41 | as soon as the experiment has finished, use the `--no-cli` parameter: |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 42 | |
| 43 | sudo minindn --no-cli ... |
| 44 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 45 | To ping only a percentage of nodes `--pct-traffic` can be set. |
| 46 | |
| 47 | sudo minindn --pct-traffic=0.5 ... |
| 48 | |
| 49 | The above command will ping only 50% of other nodes from each node. |
| 50 | The default value is 1 i.e. ping every other node. |
| 51 | |
| 52 | To move the experiment results to a results directory from the working directory |
| 53 | after the experiment is complete (either --no-cli or quit) the following option |
| 54 | can be used: |
| 55 | |
| 56 | sudo minindn --result-dir /home/mydir/result-dir ... |
| 57 | |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 58 | The included experiments are described in detail below along with additional |
| 59 | parameters that can be provided to modify the execution of the experiments. |
| 60 | |
| 61 | ### Pingall experiment |
| 62 | |
| 63 | **Scenario**: Each node in the network simultaneously pings every other node in the network at a |
| 64 | one second interval. |
| 65 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 66 | **Experiment ID**: `--pingall` |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 67 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 68 | The number of pings sent in the experiment can be configured using the `--nPings` parameter: |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 69 | |
| 70 | sudo minindn --experiment=pingall --nPings=120 |
| 71 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 72 | By default, `--nPings` is 300 for the Pingall experiment. |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 73 | |
| 74 | ### Failure experiment |
| 75 | |
| 76 | **Scenario**: Each node in the network simultaneously pings every other node in the network at a |
| 77 | one second interval. After 60 seconds, the node with the name "csu" is brought down. The node is |
| 78 | left in a failed state for 120 seconds while the other nodes continue pinging. After this period, |
| 79 | the failed node is recovered and pings are collected for an additional 90 seconds. |
| 80 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 81 | **Experiment ID**: `--failure` |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 82 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 83 | `--nPings` is 300 for the Failure experiment and cannot be modified. |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 84 | |
| 85 | ### Multiple failure experiment |
| 86 | |
| 87 | **Scenario**: Each node in the network simultaneously pings every other node in the network at a |
| 88 | one second interval. After 60 seconds, the first node in the network will be brought down and remain |
| 89 | in failed state for 60 seconds. After the failure period, the node is recovered and the network |
| 90 | is allowed to recover for 60 seconds. After the recovery period, the next node will go through this |
| 91 | failure and recovery process. Once every node in the network has gone through the failure and |
| 92 | recovery process, the experiment will end. |
| 93 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 94 | **Experiment ID**: `--multiple-failure` |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 95 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 96 | `--nPings` is dependent on the size of the topology being tested. 120 pings are scheduled for |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 97 | each node's failure/recovery period as well as an additional 60 pings for the initial collection |
| 98 | period. |
| 99 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 100 | ### MCN failure experiment |
| 101 | |
| 102 | **Scenario**: This is exactly like the failure experiment but instead of failing the node named "csu" it fails the most connected node (MCN) i.e the node with the most links. |
| 103 | |
| 104 | Experiment ID: `--failure-mcn` |
| 105 | |
| 106 | ### Experiment data |
| 107 | |
dmcoomes | 80eeea1 | 2017-10-27 12:49:10 -0500 | [diff] [blame] | 108 | The ping data is stored at `/tmp/minindn/node-name/ping-data`. |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 109 | |
dmcoomes | 80eeea1 | 2017-10-27 12:49:10 -0500 | [diff] [blame] | 110 | The ping server log is stored at `/tmp/minindn/node-name/ping-server` |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 111 | |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 112 | ## Creating custom experiments |
| 113 | |
| 114 | Mini-NDN provides a simple Python based framework which allows a user to define their own experiment |
| 115 | and run it from the command line. |
| 116 | |
| 117 | To create an experiment, follow these steps: |
| 118 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 119 | 1. Create a Python source file for the experiment in the `ndn/experiments` directory. |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 120 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 121 | e.g.) `ndn/experiments/example.py` |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 122 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 123 | 2. Derive the experiment from the `Experiment` base class defined in |
| 124 | `ndn/experiments/experiment.py`. |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 125 | |
| 126 | #!/usr/bin/python |
| 127 | |
| 128 | from ndn.experiments.experiment import Experiment |
| 129 | |
| 130 | class ExampleExperiment(Experiment): |
| 131 | def __init__(self, args): |
| 132 | Experiment.__init__(self, args) |
| 133 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 134 | 3. Override the `setup()` method to define how the experiment should be initialized |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 135 | |
| 136 | e.g.) Run an ndnping server in the background on each node |
| 137 | |
| 138 | def setup(self): |
| 139 | for host in self.net.hosts: |
| 140 | host.cmd("ndnpingserver host.name &") |
| 141 | |
| 142 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 143 | 4. Override the `run()` method to define how the experiment should behave |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 144 | |
| 145 | e.g.) Obtain the NFD status of each node and save it to file |
| 146 | |
| 147 | def run(self): |
| 148 | for host in self.net.hosts: |
Ashlesh Gawande | f932a18 | 2016-12-19 23:45:26 -0600 | [diff] [blame] | 149 | host.cmd("nfdc status report > status.txt") |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 150 | |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 151 | 5. Register the experiment with the `ExperimentManager` to make the experiment runnable from the |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 152 | command line. |
| 153 | |
| 154 | Experiment.register("example-name", ExampleExperiment) |
| 155 | |
| 156 | The experiment can then be run from the command-line using the name registered. |
| 157 | "example-name" in the above example: |
| 158 | |
| 159 | sudo minindn --experiment=example-name |
| 160 | |
| 161 | ### Full example experiment code |
| 162 | |
| 163 | #!/usr/bin/python |
| 164 | |
| 165 | from ndn.experiments.experiment import Experiment |
| 166 | |
| 167 | class ExampleExperiment(Experiment): |
| 168 | def __init__(self, args): |
| 169 | Experiment.__init__(self, args) |
| 170 | |
| 171 | def setup(self): |
| 172 | for host in self.net.hosts: |
| 173 | host.cmd("ndnpingserver host.name &") |
| 174 | |
| 175 | def run(self): |
| 176 | for host in self.net.hosts: |
Ashlesh Gawande | e626b63 | 2016-08-12 16:50:14 -0500 | [diff] [blame] | 177 | # By default status.txt would be stored |
dmcoomes | 80eeea1 | 2017-10-27 12:49:10 -0500 | [diff] [blame] | 178 | # at /tmp/minindn/host/status.txt |
Ashlesh Gawande | f932a18 | 2016-12-19 23:45:26 -0600 | [diff] [blame] | 179 | host.cmd("nfdc status report > status.txt") |
Vince Lehman | ce7f4e9 | 2015-07-14 15:22:07 -0500 | [diff] [blame] | 180 | |
| 181 | Experiment.register("example-name", ExampleExperiment) |
Ashlesh Gawande | 501d4d6 | 2017-10-25 13:12:11 -0500 | [diff] [blame] | 182 | |
| 183 | ## Passing arbitrary arguments to experiments |
| 184 | |
| 185 | One can pass any arbitrary argument to the Mini-NDN command line |
| 186 | as long as the arguments don't clash with Mini-NDN's arguments. |
| 187 | This feature allows users to pass any argument to Mini-NDN and process |
| 188 | them in an experiment without modifying Mini-NDN's core. |
| 189 | |
| 190 | Please look at `ndn/experiments/arbitrary_arguments_experiment.py` |
| 191 | to see how these arguments can be accessed. To have the experiment |
| 192 | options printed in `sudo minindn --list-experiments` when using |
| 193 | arbitrary arguments one can add the static `arguments` method as |
| 194 | shown in the aforementioned experiment. |
| 195 | |
| 196 | To run the experiment: |
| 197 | |
| 198 | sudo minindn --experiment arbitrary-arguments --ds 400 --logging false |
| 199 | |
| 200 | The experiment will print out the supplied arbitrary values which are --ds |
| 201 | and --logging. --experiment is a fixed argument of Mini-NDN. |