From e1d86d381714cb4e85f543fab0ca71f13e58767b Mon Sep 17 00:00:00 2001 From: Robert Hogan Date: Thu, 30 Sep 2010 21:41:20 +0100 Subject: [PATCH 01/26] Issues with router_get_by_nickname() https://trac.torproject.org/projects/tor/ticket/1859 There are two problems in this bug: 1. When an OP makes a .exit request specifying itself as the exit, and the exit is not yet listed, Tor gets all the routerinfos needed for the circuit but discovers in circuit_is_acceptable() that its own routerinfo is not in the routerdigest list and cannot be used. Tor then gets locked in a cycle of repeating these two steps. When gathering the routerinfos for a circuit, specifically when the exit has been chosen by .exit notation, Tor needs to apply the same rules it uses later on when deciding if it can build a circuit with those routerinfos. 2. A different bug arises in the above situation when the Tor instance's routerinfo *is* listed in the routerlist, it shares its nickname with a number of other Tor nodes, and it does not have 'Named' rights to its nickname. So for example, if (i) there are five nodes named Bob in the network, (ii) I am running one of them but am flagged as 'Unnamed' because someone else claimed the 'Bob' nickname first, and (iii) I run my Tor as both client and exit the following can happen to me: - I go to www.evil.com - I click on a link www.evil.com.bob.exit - My request will exit through my own Tor node rather than the 'Named' node Bob or any of the others. - www.evil.com now knows I am actually browsing from the same computer that is running my 'Bob' node So to solve both issues we need to ensure: - When fulfilling a .exit request we only choose a routerinfo if it exists in the routerlist, even when that routerinfo is ours. - When getting a router by nickname we only return our own router information if it is not going to be used for building a circuit. We ensure this by removing the special treatment afforded our own router in router_get_by_nickname(). This means the function will only return the routerinfo of our own router if it is in the routerlist built from authority info and has a unique nickname or is bound to a non-unique nickname. There are some uses of router_get_by_nickname() where we are looking for the router by name because of a configuration directive, specifically local declaration of NodeFamilies and EntryNodes and other routers' declaration of MyFamily. In these cases it is not at first clear if we need to continue returning our own routerinfo even if our router is not listed and/or has a non-unique nickname with the Unnamed flag. The patch treats each of these cases as follows: Other Routers' Declaration of MyFamily This happens in routerlist_add_family(). If another router declares our router in its family and our router has the Unnamed flag or is not in the routerlist yet, should we take advantage of the fact that we know our own routerinfo to add us in anyway? This patch says 'no, treat our own router just like any other'. This is a safe choice because it ensures our client has the same view of the network as other clients. We also have no good way of knowing if our router is Named or not independently of the authorities, so we have to rely on them in this. Local declaration of NodeFamilies Again, we have no way of knowing if the declaration 'NodeFamilies Bob,Alice,Ringo' refers to our router Bob or the Named router Bob, so we have to defer to the authorities and treat our own router like any other. Local declaration of NodeFamilies Again, same as above. There's also no good reason we would want our client to choose it's own router as an entry guard if it does not meet the requirements expected of any other router on the network. In order to reduce the possibility of error, the patch also replaces two instances where we were using router_get_by_nickname() with calls to router_get_by_hexdigest() where the identity digest of the router is available. --- src/or/connection_edge.c | 2 +- src/or/rendclient.c | 2 +- src/or/routerlist.c | 3 --- 3 files changed, 2 insertions(+), 5 deletions(-) diff --git a/src/or/connection_edge.c b/src/or/connection_edge.c index ded02e936..eba83ba9e 100644 --- a/src/or/connection_edge.c +++ b/src/or/connection_edge.c @@ -571,7 +571,7 @@ circuit_discard_optional_exit_enclaves(extend_info_t *info) !edge_conn->chosen_exit_retries) continue; r1 = router_get_by_nickname(edge_conn->chosen_exit_name, 0); - r2 = router_get_by_nickname(info->nickname, 0); + r2 = router_get_by_hexdigest(info->identity_digest); if (!r1 || !r2 || r1 != r2) continue; tor_assert(edge_conn->socks_request); diff --git a/src/or/rendclient.c b/src/or/rendclient.c index 7023b7f27..97fb6f0fd 100644 --- a/src/or/rendclient.c +++ b/src/or/rendclient.c @@ -810,7 +810,7 @@ rend_client_get_random_intro(const rend_data_t *rend_query) intro = smartlist_get(entry->parsed->intro_nodes, i); /* Do we need to look up the router or is the extend info complete? */ if (!intro->extend_info->onion_key) { - router = router_get_by_nickname(intro->extend_info->nickname, 0); + router = router_get_by_hexdigest(intro->extend_info->identity_digest); if (!router) { log_info(LD_REND, "Unknown router with nickname '%s'; trying another.", intro->extend_info->nickname); diff --git a/src/or/routerlist.c b/src/or/routerlist.c index 2fc80738f..962240669 100644 --- a/src/or/routerlist.c +++ b/src/or/routerlist.c @@ -1989,9 +1989,6 @@ router_get_by_nickname(const char *nickname, int warn_if_unnamed) return router_get_by_hexdigest(nickname); if (!strcasecmp(nickname, UNNAMED_ROUTER_NICKNAME)) return NULL; - if (server_mode(get_options()) && - !strcasecmp(nickname, get_options()->Nickname)) - return router_get_my_routerinfo(); maybedigest = (strlen(nickname) >= HEX_DIGEST_LEN) && (base16_decode(digest,DIGEST_LEN,nickname,HEX_DIGEST_LEN) == 0); From 7488fe5a224bf1c40d22769ff3986361d947d6f7 Mon Sep 17 00:00:00 2001 From: Robert Hogan Date: Sun, 17 Oct 2010 12:27:57 +0100 Subject: [PATCH 02/26] Issues with router_get_by_nickname() https://trac.torproject.org/projects/tor/ticket/1859 Use router_get_by_digest() instead of router_get_by_hexdigest() in circuit_discard_optional_exit_enclaves() and rend_client_get_random_intro(), per Nick's comments. Using router_get_by_digest() in rend_client_get_random_intro() will break hidden services published by Tor versions pre 0.1.2.18 and 0.2.07-alpha as they only publish by nickname. This is acceptable however as these versions only publish to authority tor26 and don't work for versions in the 0.2.2.x series anyway. --- src/or/connection_edge.c | 2 +- src/or/rendclient.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/or/connection_edge.c b/src/or/connection_edge.c index eba83ba9e..f72aa9418 100644 --- a/src/or/connection_edge.c +++ b/src/or/connection_edge.c @@ -571,7 +571,7 @@ circuit_discard_optional_exit_enclaves(extend_info_t *info) !edge_conn->chosen_exit_retries) continue; r1 = router_get_by_nickname(edge_conn->chosen_exit_name, 0); - r2 = router_get_by_hexdigest(info->identity_digest); + r2 = router_get_by_digest(info->identity_digest); if (!r1 || !r2 || r1 != r2) continue; tor_assert(edge_conn->socks_request); diff --git a/src/or/rendclient.c b/src/or/rendclient.c index 97fb6f0fd..ca3f213dc 100644 --- a/src/or/rendclient.c +++ b/src/or/rendclient.c @@ -810,7 +810,7 @@ rend_client_get_random_intro(const rend_data_t *rend_query) intro = smartlist_get(entry->parsed->intro_nodes, i); /* Do we need to look up the router or is the extend info complete? */ if (!intro->extend_info->onion_key) { - router = router_get_by_hexdigest(intro->extend_info->identity_digest); + router = router_get_by_digest(intro->extend_info->identity_digest); if (!router) { log_info(LD_REND, "Unknown router with nickname '%s'; trying another.", intro->extend_info->nickname); From 5cd7953a2112d6ae8702c741d73b8b6a2d0c5251 Mon Sep 17 00:00:00 2001 From: Robert Hogan Date: Sun, 17 Oct 2010 15:12:25 +0100 Subject: [PATCH 03/26] Issues with router_get_by_nickname() (3) Add changes file --- changes/bug1859 | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 changes/bug1859 diff --git a/changes/bug1859 b/changes/bug1859 new file mode 100644 index 000000000..5b139f357 --- /dev/null +++ b/changes/bug1859 @@ -0,0 +1,9 @@ + o Minor bugfixes: + - Bring the logic that gathers routerinfos and assesses the + acceptability of circuits into line. This prevents a Tor OP from getting + locked in a cycle of choosing its local OR as an exit for a path (due to + a .exit request) and then rejecting the circuit because its OR is not + listed yet. Also prevent Tor clients from using an OR running in the same + instance as an exit (due to a .exit request) if the OR does not meet the + same requirements expected of an OR running elsewhere. + Fixes bug 1859; bugfix on 0.2.0-alpha. From 2392290c1887b732fd2309e57ce04ab71b9b4e69 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Thu, 21 Oct 2010 11:08:15 -0400 Subject: [PATCH 04/26] Fix a remaining bug in Robert's bug1859 fix. When intro->extend_info is created for an introduction point, it only starts out with a nickname, not necessarily an identity digest. Thus, doing router_get_by_digest isn't necessarily safe. --- src/or/rendclient.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/or/rendclient.c b/src/or/rendclient.c index ca3f213dc..1f253c10b 100644 --- a/src/or/rendclient.c +++ b/src/or/rendclient.c @@ -810,7 +810,10 @@ rend_client_get_random_intro(const rend_data_t *rend_query) intro = smartlist_get(entry->parsed->intro_nodes, i); /* Do we need to look up the router or is the extend info complete? */ if (!intro->extend_info->onion_key) { - router = router_get_by_digest(intro->extend_info->identity_digest); + if (tor_digest_is_zero(intro->extend_info->identity_digest)) + router = router_get_by_hexdigest(intro->extend_info->nickname); + else + router = router_get_by_digest(intro->extend_info->identity_digest); if (!router) { log_info(LD_REND, "Unknown router with nickname '%s'; trying another.", intro->extend_info->nickname); From 8dd4ecd14e6e6e886c76c8464595d4f903447c42 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Mon, 10 Jan 2011 12:57:44 -0500 Subject: [PATCH 05/26] Turn the loop bodies of rep_hist_save/load_state into functions The trick of looping from i=0..4 , switching on i to set up some variables, then running some common code is much better expressed by just calling a function 4 times with 4 sets of arguments. This should make the code a little easier to follow and maintain here. --- src/or/rephist.c | 241 ++++++++++++++++++++++------------------------- 1 file changed, 114 insertions(+), 127 deletions(-) diff --git a/src/or/rephist.c b/src/or/rephist.c index d85a8b7c3..c220426d5 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -1392,7 +1392,7 @@ rep_hist_bandwidth_assess(void) * It returns the number of bytes written. */ static size_t -rep_hist_fill_bandwidth_history(char *buf, size_t len, bw_array_t *b) +rep_hist_fill_bandwidth_history(char *buf, size_t len, const bw_array_t *b) { char *cp = buf; int i, n; @@ -1484,161 +1484,148 @@ rep_hist_get_bandwidth_lines(void) return buf; } +/** Write a single bw_warray_t from its Values, Ends, and Interval entries + * from a state. */ +static void +rep_hist_update_bwhist_state_section(or_state_t *state, + const bw_array_t *b, + smartlist_t **s_values, + time_t *s_begins, + int *s_interval) +{ + char buf[20*NUM_TOTALS + 1], *cp; + + if (*s_values) { + SMARTLIST_FOREACH(*s_values, char *, val, tor_free(val)); + smartlist_free(*s_values); + } + if (! server_mode(get_options())) { + /* Clients don't need to store bandwidth history persistently; + * force these values to the defaults. */ + /* FFFF we should pull the default out of config.c's state table, + * so we don't have two defaults. */ + if (*s_begins != 0 || *s_interval != 900) { + time_t now = time(NULL); + time_t save_at = get_options()->AvoidDiskWrites ? now+3600 : now+600; + or_state_mark_dirty(state, save_at); + } + *s_begins = 0; + *s_interval = 900; + *s_values = smartlist_create(); + return; + } + *s_begins = b->next_period; + *s_interval = NUM_SECS_BW_SUM_INTERVAL; + cp = buf; + cp += rep_hist_fill_bandwidth_history(cp, sizeof(buf), b); + tor_snprintf(cp, sizeof(buf)-(cp-buf), + cp == buf ? U64_FORMAT : ","U64_FORMAT, + U64_PRINTF_ARG(b->total_in_period)); + *s_values = smartlist_create(); + if (server_mode(get_options())) + smartlist_split_string(*s_values, buf, ",", SPLIT_SKIP_SPACE, 0); +} + /** Update state with the newest bandwidth history. */ void rep_hist_update_state(or_state_t *state) { - int len, r; - char *buf, *cp; - smartlist_t **s_values = NULL; - time_t *s_begins = NULL; - int *s_interval = NULL; - bw_array_t *b = NULL; +#define UPDATE(arrname,st) \ + rep_hist_update_bwhist_state_section(state,\ + (arrname),\ + &state->BWHistory ## st ## Values, \ + &state->BWHistory ## st ## Ends, \ + &state->BWHistory ## st ## Interval) - len = 20*NUM_TOTALS+1; - buf = tor_malloc_zero(len); + UPDATE(write_array, Write); + UPDATE(read_array, Read); + UPDATE(dir_write_array, DirWrite); + UPDATE(dir_read_array, DirRead); - for (r=0;r<4;++r) { - switch (r) { - case 0: - b = write_array; - s_begins = &state->BWHistoryWriteEnds; - s_interval = &state->BWHistoryWriteInterval; - s_values = &state->BWHistoryWriteValues; - break; - case 1: - b = read_array; - s_begins = &state->BWHistoryReadEnds; - s_interval = &state->BWHistoryReadInterval; - s_values = &state->BWHistoryReadValues; - break; - case 2: - b = dir_write_array; - s_begins = &state->BWHistoryDirWriteEnds; - s_interval = &state->BWHistoryDirWriteInterval; - s_values = &state->BWHistoryDirWriteValues; - break; - case 3: - b = dir_read_array; - s_begins = &state->BWHistoryDirReadEnds; - s_interval = &state->BWHistoryDirReadInterval; - s_values = &state->BWHistoryDirReadValues; - break; - } - if (*s_values) { - SMARTLIST_FOREACH(*s_values, char *, val, tor_free(val)); - smartlist_free(*s_values); - } - if (! server_mode(get_options())) { - /* Clients don't need to store bandwidth history persistently; - * force these values to the defaults. */ - /* FFFF we should pull the default out of config.c's state table, - * so we don't have two defaults. */ - if (*s_begins != 0 || *s_interval != 900) { - time_t now = time(NULL); - time_t save_at = get_options()->AvoidDiskWrites ? now+3600 : now+600; - or_state_mark_dirty(state, save_at); - } - *s_begins = 0; - *s_interval = 900; - *s_values = smartlist_create(); - continue; - } - *s_begins = b->next_period; - *s_interval = NUM_SECS_BW_SUM_INTERVAL; - cp = buf; - cp += rep_hist_fill_bandwidth_history(cp, len, b); - tor_snprintf(cp, len-(cp-buf), cp == buf ? U64_FORMAT : ","U64_FORMAT, - U64_PRINTF_ARG(b->total_in_period)); - *s_values = smartlist_create(); - if (server_mode(get_options())) - smartlist_split_string(*s_values, buf, ",", SPLIT_SKIP_SPACE, 0); - } - tor_free(buf); if (server_mode(get_options())) { - or_state_mark_dirty(get_or_state(), time(NULL)+(2*3600)); + or_state_mark_dirty(state, time(NULL)+(2*3600)); } +#undef UPDATE } -/** Set bandwidth history from our saved state. */ -int -rep_hist_load_state(or_state_t *state, char **err) +/** Load a single bw_warray_t from its Values, Ends, and Interval entries from + * a state. */ +static int +rep_hist_load_bwhist_state_section(bw_array_t *b, + const smartlist_t *s_values, + const time_t s_begins, + const int s_interval) { - time_t s_begins = 0, start; time_t now = time(NULL); + int retval = 0; + time_t start; + uint64_t v; - int r,i,ok; - int all_ok = 1; - int s_interval = 0; - smartlist_t *s_values = NULL; - bw_array_t *b = NULL; + int i,ok; - /* Assert they already have been malloced */ - tor_assert(read_array && write_array); - - for (r=0;r<4;++r) { - switch (r) { - case 0: - b = write_array; - s_begins = state->BWHistoryWriteEnds; - s_interval = state->BWHistoryWriteInterval; - s_values = state->BWHistoryWriteValues; - break; - case 1: - b = read_array; - s_begins = state->BWHistoryReadEnds; - s_interval = state->BWHistoryReadInterval; - s_values = state->BWHistoryReadValues; - break; - case 2: - b = dir_write_array; - s_begins = state->BWHistoryDirWriteEnds; - s_interval = state->BWHistoryDirWriteInterval; - s_values = state->BWHistoryDirWriteValues; - break; - case 3: - b = dir_read_array; - s_begins = state->BWHistoryDirReadEnds; - s_interval = state->BWHistoryDirReadInterval; - s_values = state->BWHistoryDirReadValues; - break; - } - if (s_values && s_begins >= now - NUM_SECS_BW_SUM_INTERVAL*NUM_TOTALS) { - start = s_begins - s_interval*(smartlist_len(s_values)); - if (start > now) - continue; - b->cur_obs_time = start; - b->next_period = start + NUM_SECS_BW_SUM_INTERVAL; - SMARTLIST_FOREACH(s_values, char *, cp, { + if (s_values && s_begins >= now - NUM_SECS_BW_SUM_INTERVAL*NUM_TOTALS) { + start = s_begins - s_interval*(smartlist_len(s_values)); + if (start > now) + return 0; + b->cur_obs_time = start; + b->next_period = start + NUM_SECS_BW_SUM_INTERVAL; + SMARTLIST_FOREACH_BEGIN(s_values, const char *, cp) { v = tor_parse_uint64(cp, 10, 0, UINT64_MAX, &ok, NULL); if (!ok) { - all_ok=0; + retval = -1; log_notice(LD_HIST, "Could not parse '%s' into a number.'", cp); } if (start < now) { add_obs(b, start, v); start += NUM_SECS_BW_SUM_INTERVAL; } - }); - } - - /* Clean up maxima and observed */ - /* Do we really want to zero this for the purpose of max capacity? */ - for (i=0; iobs[i] = 0; - } - b->total_obs = 0; - for (i=0; imaxima[i] = 0; - } - b->max_total = 0; + } SMARTLIST_FOREACH_END(cp); } + /* Clean up maxima and observed */ + /* Do we really want to zero this for the purpose of max capacity? */ + for (i=0; iobs[i] = 0; + } + b->total_obs = 0; + for (i=0; imaxima[i] = 0; + } + b->max_total = 0; + + return retval; +} + +/** Set bandwidth history from our saved state. */ +int +rep_hist_load_state(or_state_t *state, char **err) +{ + int all_ok = 1; + + /* Assert they already have been malloced */ + tor_assert(read_array && write_array); + tor_assert(dir_read_array && dir_write_array); + +#define LOAD(arrname,st) \ + if (rep_hist_load_bwhist_state_section( \ + (arrname), \ + state->BWHistory ## st ## Values, \ + state->BWHistory ## st ## Ends, \ + state->BWHistory ## st ## Interval)<0) \ + all_ok = 0 + + LOAD(write_array, Write); + LOAD(read_array, Read); + LOAD(dir_write_array, DirWrite); + LOAD(dir_read_array, DirRead); + +#undef LOAD if (!all_ok) { *err = tor_strdup("Parsing of bandwidth history values failed"); /* and create fresh arrays */ tor_free(read_array); tor_free(write_array); + read_array = bw_array_new(); write_array = bw_array_new(); return -1; From 7e1502c0d1aa0875fba0b26ddcf021bdfa5e11a0 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Mon, 10 Jan 2011 13:06:50 -0500 Subject: [PATCH 06/26] Fix bug when parsing bwhist with unexpected Interval Previously, our state parsing code would fail to parse a bwhist correctly if the Interval was anything other than the default hardcoded 15 minutes. This change makes the parsing less incorrect, though the resulting history array might get strange values in it if the intervals don't match the one we're using. (That is, if stuff was generated in 15 minute intervals, and we read it into an array that expects 30 minute intervals, we're fine, since values can be combined pairwise. But if we generate data at 30 minute intervals and read it into 15 minute intervals, alternating buckets will be empty.) Bugfix on 0.1.1.11-alpha. --- changes/1863_bwhist | 5 +++++ src/or/rephist.c | 6 +++++- 2 files changed, 10 insertions(+), 1 deletion(-) create mode 100644 changes/1863_bwhist diff --git a/changes/1863_bwhist b/changes/1863_bwhist new file mode 100644 index 000000000..c1d4d866c --- /dev/null +++ b/changes/1863_bwhist @@ -0,0 +1,5 @@ + o Minor bugfixes + - Fix a bug in banwidth history state parsing that could have been + triggered if a future version of Tor ever changed the timing + granularity at which bandwidth history is measured. Bugfix on + Tor 0.1.1.11-alpha. diff --git a/src/or/rephist.c b/src/or/rephist.c index c220426d5..5a3a15edb 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -1165,6 +1165,8 @@ rep_hist_load_mtbf_data(time_t now) * totals? */ #define NUM_SECS_ROLLING_MEASURE 10 /** How large are the intervals for which we track and report bandwidth use? */ +/* XXXX Watch out! Before Tor 0.2.2.21-alpha, using any other value here would + * generate an unparseable state file. */ #define NUM_SECS_BW_SUM_INTERVAL (15*60) /** How far in the past do we remember and publish bandwidth use? */ #define NUM_SECS_BW_SUM_IS_VALID (24*60*60) @@ -1577,7 +1579,9 @@ rep_hist_load_bwhist_state_section(bw_array_t *b, } if (start < now) { add_obs(b, start, v); - start += NUM_SECS_BW_SUM_INTERVAL; + /* This will result in some fairly choppy history if s_interval + * is notthe same as NUM_SECS_BW_SUM_INTERVAL. XXXX */ + start += s_interval; } } SMARTLIST_FOREACH_END(cp); } From 28844c8403f16f184c185798c78dcd6a959450a3 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Mon, 10 Jan 2011 13:15:51 -0500 Subject: [PATCH 07/26] On bwhist state load failure, clear dir_read/write hist too --- changes/1863_bwhist | 5 +++++ src/or/rephist.c | 13 +++++++------ 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/changes/1863_bwhist b/changes/1863_bwhist index c1d4d866c..b9e8ef4c5 100644 --- a/changes/1863_bwhist +++ b/changes/1863_bwhist @@ -3,3 +3,8 @@ triggered if a future version of Tor ever changed the timing granularity at which bandwidth history is measured. Bugfix on Tor 0.1.1.11-alpha. + - Correctly clear out dir_read/dir_write history when there is an + error parsing any bw history value from the state file. Bugfix on + Tor 0.2.2.15-alpha. + + diff --git a/src/or/rephist.c b/src/or/rephist.c index 5a3a15edb..f0dd45128 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -1293,10 +1293,15 @@ static bw_array_t *dir_read_array = NULL; directory protocol. */ static bw_array_t *dir_write_array = NULL; -/** Set up [dir-]read_array and [dir-]write_array. */ +/** Set up [dir-]read_array and [dir-]write_array, freeing them if they alrady + * exist. */ static void bw_arrays_init(void) { + tor_free(read_array); + tor_free(write_array); + tor_free(dir_read_array); + tor_free(dir_write_array); read_array = bw_array_new(); write_array = bw_array_new(); dir_read_array = bw_array_new(); @@ -1627,11 +1632,7 @@ rep_hist_load_state(or_state_t *state, char **err) if (!all_ok) { *err = tor_strdup("Parsing of bandwidth history values failed"); /* and create fresh arrays */ - tor_free(read_array); - tor_free(write_array); - - read_array = bw_array_new(); - write_array = bw_array_new(); + bw_arrays_init(); return -1; } return 0; From 105b94b75b51376128037cab05b930c4e4c15497 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Mon, 10 Jan 2011 14:11:22 -0500 Subject: [PATCH 08/26] Add Maxima lists to bandwidth state. Right now, Tor routers don't save the maxima values from the bw_history_t between sessions. That's no good, since we use those values to determine bandwidth. This code adds a new BWHist.*Maximum set of values to the state file. If they're not present, we estimate them by taking the observed total bandwidth and dividing it by the period length, which provides a lower bound. This should fix bug 1863. I'm calling it a feature. --- changes/1863_bwhist | 8 +++++- src/or/config.c | 4 +++ src/or/or.h | 8 +++++- src/or/rephist.c | 65 +++++++++++++++++++++++++++++++++------------ 4 files changed, 66 insertions(+), 19 deletions(-) diff --git a/changes/1863_bwhist b/changes/1863_bwhist index b9e8ef4c5..b94250906 100644 --- a/changes/1863_bwhist +++ b/changes/1863_bwhist @@ -1,3 +1,10 @@ + o Minor features + - Servers now save observed maximum bandwidth throughput rates + to their state file (along with total usage, which was already + saved) so that they can determine their correct estimated + bandwidth on restart. Resolves bug 1863, where Tor servers + would reset their estimated bandwidth to 0 after restarting. + o Minor bugfixes - Fix a bug in banwidth history state parsing that could have been triggered if a future version of Tor ever changed the timing @@ -7,4 +14,3 @@ error parsing any bw history value from the state file. Bugfix on Tor 0.2.2.15-alpha. - diff --git a/src/or/config.c b/src/or/config.c index 17d776e71..78978dc08 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -443,15 +443,19 @@ static config_var_t _state_vars[] = { V(BWHistoryReadEnds, ISOTIME, NULL), V(BWHistoryReadInterval, UINT, "900"), V(BWHistoryReadValues, CSV, ""), + V(BWHistoryReadMaxima, CSV, ""), V(BWHistoryWriteEnds, ISOTIME, NULL), V(BWHistoryWriteInterval, UINT, "900"), V(BWHistoryWriteValues, CSV, ""), + V(BWHistoryWriteMaxima, CSV, ""), V(BWHistoryDirReadEnds, ISOTIME, NULL), V(BWHistoryDirReadInterval, UINT, "900"), V(BWHistoryDirReadValues, CSV, ""), + V(BWHistoryDirReadMaxima, CSV, ""), V(BWHistoryDirWriteEnds, ISOTIME, NULL), V(BWHistoryDirWriteInterval, UINT, "900"), V(BWHistoryDirWriteValues, CSV, ""), + V(BWHistoryDirWriteMaxima, CSV, ""), V(TorVersion, STRING, NULL), diff --git a/src/or/or.h b/src/or/or.h index cb36126d9..e0f8babe7 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -2856,19 +2856,25 @@ typedef struct { * bandwidth usage. The "Interval" fields hold the granularity, in seconds, * of the entries of Values. The "Values" lists hold decimal string * representations of the number of bytes read or written in each - * interval. */ + * interval. The "Maxima" list holds decimal strings describing the highest + * rate achieved during the interval. + */ time_t BWHistoryReadEnds; int BWHistoryReadInterval; smartlist_t *BWHistoryReadValues; + smartlist_t *BWHistoryReadMaxima; time_t BWHistoryWriteEnds; int BWHistoryWriteInterval; smartlist_t *BWHistoryWriteValues; + smartlist_t *BWHistoryWriteMaxima; time_t BWHistoryDirReadEnds; int BWHistoryDirReadInterval; smartlist_t *BWHistoryDirReadValues; + smartlist_t *BWHistoryDirReadMaxima; time_t BWHistoryDirWriteEnds; int BWHistoryDirWriteInterval; smartlist_t *BWHistoryDirWriteValues; + smartlist_t *BWHistoryDirWriteMaxima; /** Build time histogram */ config_line_t * BuildtimeHistogram; diff --git a/src/or/rephist.c b/src/or/rephist.c index f0dd45128..a3a2bab95 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -1261,8 +1261,12 @@ add_obs(bw_array_t *b, time_t when, uint64_t n) /* If we're currently adding observations for an earlier second than * 'when', advance b->cur_obs_time and b->cur_obs_idx by an * appropriate number of seconds, and do all the other housekeeping */ - while (when>b->cur_obs_time) + while (when>b->cur_obs_time) { + /* Doing this one second at a time is potentially inefficient, if we start + with a state file that is very old. Fortunately, it doesn't seem to + show up in profiles, so we can just ignore it for now. */ advance_obs(b); + } b->obs[b->cur_obs_idx] += n; b->total_in_period += n; @@ -1497,15 +1501,21 @@ static void rep_hist_update_bwhist_state_section(or_state_t *state, const bw_array_t *b, smartlist_t **s_values, + smartlist_t **s_maxima, time_t *s_begins, int *s_interval) { - char buf[20*NUM_TOTALS + 1], *cp; + char *cp; + int i,j; if (*s_values) { SMARTLIST_FOREACH(*s_values, char *, val, tor_free(val)); smartlist_free(*s_values); } + if (*s_maxima) { + SMARTLIST_FOREACH(*s_values, char *, val, tor_free(val)); + smartlist_free(*s_maxima); + } if (! server_mode(get_options())) { /* Clients don't need to store bandwidth history persistently; * force these values to the defaults. */ @@ -1519,18 +1529,30 @@ rep_hist_update_bwhist_state_section(or_state_t *state, *s_begins = 0; *s_interval = 900; *s_values = smartlist_create(); + *s_maxima = smartlist_create(); return; } *s_begins = b->next_period; *s_interval = NUM_SECS_BW_SUM_INTERVAL; - cp = buf; - cp += rep_hist_fill_bandwidth_history(cp, sizeof(buf), b); - tor_snprintf(cp, sizeof(buf)-(cp-buf), - cp == buf ? U64_FORMAT : ","U64_FORMAT, - U64_PRINTF_ARG(b->total_in_period)); + *s_values = smartlist_create(); - if (server_mode(get_options())) - smartlist_split_string(*s_values, buf, ",", SPLIT_SKIP_SPACE, 0); + *s_maxima = smartlist_create(); + /* Set i to first position in circular array */ + i = (b->num_maxes_set <= b->next_max_idx) ? 0 : b->next_max_idx; + for (j=0; j < b->num_maxes_set; ++j,++i) { + uint64_t maxval; + if (i > NUM_TOTALS) + i = 0; + tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(b->totals[i] & ~0x3ff)); + smartlist_add(*s_values, cp); + maxval = b->maxima[i] / NUM_SECS_ROLLING_MEASURE; + tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(maxval & ~0x3ff)); + smartlist_add(*s_maxima, cp); + } + tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(b->total_in_period & ~0x3ff)); + smartlist_add(*s_values, cp); + tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(b->max_total & ~0x3ff)); + smartlist_add(*s_maxima, cp); } /** Update state with the newest bandwidth history. */ @@ -1541,6 +1563,7 @@ rep_hist_update_state(or_state_t *state) rep_hist_update_bwhist_state_section(state,\ (arrname),\ &state->BWHistory ## st ## Values, \ + &state->BWHistory ## st ## Maxima, \ &state->BWHistory ## st ## Ends, \ &state->BWHistory ## st ## Interval) @@ -1560,6 +1583,7 @@ rep_hist_update_state(or_state_t *state) static int rep_hist_load_bwhist_state_section(bw_array_t *b, const smartlist_t *s_values, + const smartlist_t *s_maxima, const time_t s_begins, const int s_interval) { @@ -1567,8 +1591,9 @@ rep_hist_load_bwhist_state_section(bw_array_t *b, int retval = 0; time_t start; - uint64_t v; - int i,ok; + uint64_t v, mv; + int i,ok,ok_m; + int have_maxima = (smartlist_len(s_values) == smartlist_len(s_maxima)); if (s_values && s_begins >= now - NUM_SECS_BW_SUM_INTERVAL*NUM_TOTALS) { start = s_begins - s_interval*(smartlist_len(s_values)); @@ -1578,12 +1603,22 @@ rep_hist_load_bwhist_state_section(bw_array_t *b, b->next_period = start + NUM_SECS_BW_SUM_INTERVAL; SMARTLIST_FOREACH_BEGIN(s_values, const char *, cp) { v = tor_parse_uint64(cp, 10, 0, UINT64_MAX, &ok, NULL); - if (!ok) { + if (have_maxima) { + const char *maxstr = smartlist_get(s_maxima, cp_sl_idx); + mv = tor_parse_uint64(maxstr, 10, 0, UINT64_MAX, &ok_m, NULL); + mv *= NUM_SECS_ROLLING_MEASURE; + } else { + /* No maxima known; guess average rate to be conservative. */ + mv = v / s_interval; + } + if (!ok || !ok_m) { retval = -1; log_notice(LD_HIST, "Could not parse '%s' into a number.'", cp); } + if (start < now) { add_obs(b, start, v); + b->max_total = mv; /* This will result in some fairly choppy history if s_interval * is notthe same as NUM_SECS_BW_SUM_INTERVAL. XXXX */ start += s_interval; @@ -1592,15 +1627,10 @@ rep_hist_load_bwhist_state_section(bw_array_t *b, } /* Clean up maxima and observed */ - /* Do we really want to zero this for the purpose of max capacity? */ for (i=0; iobs[i] = 0; } b->total_obs = 0; - for (i=0; imaxima[i] = 0; - } - b->max_total = 0; return retval; } @@ -1619,6 +1649,7 @@ rep_hist_load_state(or_state_t *state, char **err) if (rep_hist_load_bwhist_state_section( \ (arrname), \ state->BWHistory ## st ## Values, \ + state->BWHistory ## st ## Maxima, \ state->BWHistory ## st ## Ends, \ state->BWHistory ## st ## Interval)<0) \ all_ok = 0 From 80253ba4d77a95ab14ee80dbc4eea88bb264124c Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Mon, 10 Jan 2011 15:13:07 -0500 Subject: [PATCH 09/26] fix some spelling in rephist comments --- src/or/rephist.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/or/rephist.c b/src/or/rephist.c index a3a2bab95..265fff8a3 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -1297,8 +1297,8 @@ static bw_array_t *dir_read_array = NULL; directory protocol. */ static bw_array_t *dir_write_array = NULL; -/** Set up [dir-]read_array and [dir-]write_array, freeing them if they alrady - * exist. */ +/** Set up [dir-]read_array and [dir-]write_array, freeing them if they + * already exist. */ static void bw_arrays_init(void) { @@ -1495,8 +1495,8 @@ rep_hist_get_bandwidth_lines(void) return buf; } -/** Write a single bw_warray_t from its Values, Ends, and Interval entries - * from a state. */ +/** Write a single bw_array_t into the Values, Ends, Interval, and Maximum + * entries of an or_state_t. */ static void rep_hist_update_bwhist_state_section(or_state_t *state, const bw_array_t *b, @@ -1578,8 +1578,8 @@ rep_hist_update_state(or_state_t *state) #undef UPDATE } -/** Load a single bw_warray_t from its Values, Ends, and Interval entries from - * a state. */ +/** Load a single bw_array_t from its Values, Ends, Maxima, and Interval + * entries in an or_state_t. */ static int rep_hist_load_bwhist_state_section(bw_array_t *b, const smartlist_t *s_values, From a1073ee956021ead19d30c2151510dbaced416a8 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Tue, 25 Jan 2011 14:08:13 -0500 Subject: [PATCH 10/26] Simplest fix to bug2402: do not include SVN versions When we stopped using svn, 0.2.1.x lost the ability to notice its svn revision and report it in the version number. However, it kept looking at the micro-revision.i file... so if you switched to master, built tor, then switched to 0.2.1.x, you'd get a micro-revision.i file from master reported as an SVN tag. This patch takes out the "include the svn tag" logic entirely. Bugfix on 0.2.1.15-rc; fixes bug 2402. --- changes/bug2402 | 4 ++++ src/or/config.c | 11 +---------- 2 files changed, 5 insertions(+), 10 deletions(-) create mode 100644 changes/bug2402 diff --git a/changes/bug2402 b/changes/bug2402 new file mode 100644 index 000000000..f16f6773e --- /dev/null +++ b/changes/bug2402 @@ -0,0 +1,4 @@ + o Minor bugfixes (build) + - Do not include Git version tags as though they were SVN tags when + generating a tarball from inside a repository that has switched between + branches. Bugfix on 0.2.1.15-rc; fixes bug 2402. diff --git a/src/or/config.c b/src/or/config.c index f8cfd29f8..c5d654078 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -827,16 +827,7 @@ static char *_version = NULL; const char * get_version(void) { - if (_version == NULL) { - if (strlen(tor_svn_revision)) { - size_t len = strlen(VERSION)+strlen(tor_svn_revision)+8; - _version = tor_malloc(len); - tor_snprintf(_version, len, "%s (r%s)", VERSION, tor_svn_revision); - } else { - _version = tor_strdup(VERSION); - } - } - return _version; + return VERSION; } /** Release additional memory allocated in options From 23f8bedddb1b3497ea8bbafef90ae24893be9a9e Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Tue, 25 Jan 2011 15:02:36 -0500 Subject: [PATCH 11/26] Add manpage entry for logging domains Fixes issue 2215. --- changes/log_domains | 5 +++++ doc/tor.1.txt | 23 +++++++++++++++++++++++ 2 files changed, 28 insertions(+) create mode 100644 changes/log_domains diff --git a/changes/log_domains b/changes/log_domains new file mode 100644 index 000000000..f4cfe6f4d --- /dev/null +++ b/changes/log_domains @@ -0,0 +1,5 @@ + o Documentation + + - Add documentation for configuring logging at different severities in + different log domains. We've had this feature since 0.2.1.1-alpha, but + for some reason it never made it into the manpage. Fixes bug 2215. diff --git a/doc/tor.1.txt b/doc/tor.1.txt index c8608eb84..9a7ee07e7 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -323,6 +323,29 @@ Other options can be specified either on the command-line (--option Messages are sent to all the logs that match their severity level. +**Log** **[**__domain__,...**]**__minSeverity__[-__maxSeverity__] ... **file** __FILENAME__ + + +**Log** **[**__domain__,...**]**__minSeverity__[-__maxSeverity__] ... **stderr**|**stdout**|**syslog** :: + As above, but select messages by range of log severity __and__ by a + set of "logging domains". Each logging domain corresponds to an area of + functionality inside Tor. You can specify any number of severity ranges + for a single log statement, each of them prefixed by a comma-separated + list of logging domains. You can prefix a domain with ~ to indicate + negation, and use * to indicate "all domains". If you specify a severity + range without a list of domains, it matches all domains. + + + + This is an advanced feature which is most useful for debugging one or two + of Tor's subsystems at a time. + + + + The currently recognized domains are: general, crypto, net, config, fs, + protocol, mm, http, app, control, circ, rend, bug, dir, dirserv, or, edge, + acct, hist, and handshake. Domain names are case-insensitive. + + + + For example, "`Log [handshake]debug [*,~net,~mm]info notice stdout`" sends + to stdout: all handshake messages of any severity, all info-and-higher + messages from domains other than networking and memory management, and all + messages of severity notice or higher. + **OutboundBindAddress** __IP__:: Make all outbound connections originate from the IP address specified. This is only useful when you have multiple network interfaces, and you want all From e261a1a3e6513eeafa1b53b83ebfec7f5d834a39 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Tue, 25 Jan 2011 15:03:36 -0500 Subject: [PATCH 12/26] Simplify syntax for negated log domains Previously if you wanted to say "All messages except network messages", you needed to say "[*,~net]" and if you said "[~net]" by mistake, you would get no messages at all. Now, if you say "[~net]", you get everything except networking messages. --- changes/log_domains | 6 +++++- doc/tor.1.txt | 2 +- src/common/log.c | 5 ++++- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/changes/log_domains b/changes/log_domains index f4cfe6f4d..939b9a9e7 100644 --- a/changes/log_domains +++ b/changes/log_domains @@ -1,5 +1,9 @@ - o Documentation + o Minor features + - Make it simpler to specify "All log domains except for A and B". + Previously you needed to say "[*,~A,~B]". Now you can just say + "[~A,~B]". + o Documentation - Add documentation for configuring logging at different severities in different log domains. We've had this feature since 0.2.1.1-alpha, but for some reason it never made it into the manpage. Fixes bug 2215. diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 9a7ee07e7..17cdf37ac 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -341,7 +341,7 @@ Other options can be specified either on the command-line (--option protocol, mm, http, app, control, circ, rend, bug, dir, dirserv, or, edge, acct, hist, and handshake. Domain names are case-insensitive. + + - For example, "`Log [handshake]debug [*,~net,~mm]info notice stdout`" sends + For example, "`Log [handshake]debug [~net,~mm]info notice stdout`" sends to stdout: all handshake messages of any severity, all info-and-higher messages from domains other than networking and memory management, and all messages of severity notice or higher. diff --git a/src/common/log.c b/src/common/log.c index 2fef2cc5d..28fca4eac 100644 --- a/src/common/log.c +++ b/src/common/log.c @@ -919,7 +919,10 @@ parse_log_severity_config(const char **cfg_ptr, smartlist_free(domains_list); if (err) return -1; - domains &= ~neg_domains; + if (domains == 0 && neg_domains) + domains = ~neg_domains; + else + domains &= ~neg_domains; cfg = eat_whitespace(closebracket+1); } else { ++got_an_unqualified_range; From 89ee779f928af4ad31ad6d45936c6a386e263584 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Tue, 25 Jan 2011 15:53:15 -0500 Subject: [PATCH 13/26] Add a torrc option to report log domains --- changes/log_domains | 3 ++ doc/tor.1.txt | 5 +++ src/common/log.c | 76 ++++++++++++++++++++++++++++++++++++--------- src/common/torlog.h | 1 + src/or/config.c | 7 ++++- src/or/or.h | 3 ++ 6 files changed, 80 insertions(+), 15 deletions(-) diff --git a/changes/log_domains b/changes/log_domains index 939b9a9e7..7fc0506cd 100644 --- a/changes/log_domains +++ b/changes/log_domains @@ -2,6 +2,9 @@ - Make it simpler to specify "All log domains except for A and B". Previously you needed to say "[*,~A,~B]". Now you can just say "[~A,~B]". + - Add a LogMessageDomains option to include the domains of log messages + along with the messages. Without this, there's no way to use + log domains without reading the source or doing a lot of guessing o Documentation - Add documentation for configuring logging at different severities in diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 17cdf37ac..a4ab0d921 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -346,6 +346,11 @@ Other options can be specified either on the command-line (--option messages from domains other than networking and memory management, and all messages of severity notice or higher. +**LogMessageDomains** **0**|**1**:: + If 1, Tor includes message domains with each log message. Every log + message currently has at least one domain; most currently have exactly + one. This doesn't affect controller log messages. (Default: 0) + **OutboundBindAddress** __IP__:: Make all outbound connections originate from the IP address specified. This is only useful when you have multiple network interfaces, and you want all diff --git a/src/common/log.c b/src/common/log.c index 28fca4eac..f58b05b1b 100644 --- a/src/common/log.c +++ b/src/common/log.c @@ -97,6 +97,9 @@ static int log_mutex_initialized = 0; /** Linked list of logfile_t. */ static logfile_t *logfiles = NULL; +/** Boolean: do we report logging domains? */ +static int log_domains_are_logged = 0; + #ifdef HAVE_SYSLOG_H /** The number of open syslog log handlers that we have. When this reaches 0, * we can close our connection to the syslog facility. */ @@ -126,6 +129,9 @@ int _log_global_min_severity = LOG_NOTICE; static void delete_log(logfile_t *victim); static void close_log(logfile_t *victim); +static char *domain_to_string(log_domain_mask_t domain, + char *buf, size_t buflen); + /** Name of the application: used to generate the message we write at the * start of each new log. */ static char *appname = NULL; @@ -218,12 +224,22 @@ format_msg(char *buf, size_t buf_len, int r; char *end_of_prefix; - assert(buf_len >= 2); /* prevent integer underflow */ + assert(buf_len >= 16); /* prevent integer underflow and general stupidity */ buf_len -= 2; /* subtract 2 characters so we have room for \n\0 */ n = _log_prefix(buf, buf_len, severity); end_of_prefix = buf+n; + if (log_domains_are_logged) { + char *cp = buf+n; + *cp++ = '{'; + cp = domain_to_string(domain, cp, (buf+buf_len-cp)); + *cp++ = '}'; + *cp++ = ' '; + end_of_prefix = cp; + n = cp-buf; + } + if (funcname && should_log_function_name(domain, severity)) { r = tor_snprintf(buf+n, buf_len-n, "%s(): ", funcname); if (r<0) @@ -305,6 +321,7 @@ logv(int severity, log_domain_mask_t domain, const char *funcname, &msg_len); formatted = 1; } + if (lf->is_syslog) { #ifdef HAVE_SYSLOG_H char *m = end_of_prefix; @@ -563,8 +580,7 @@ add_stream_log_impl(const log_severity_list_t *severity, * to fd. Steals a reference to severity; the caller must * not use it after calling this function. */ void -add_stream_log(const log_severity_list_t *severity, - const char *name, int fd) +add_stream_log(const log_severity_list_t *severity, const char *name, int fd) { LOCK_LOGS(); add_stream_log_impl(severity, name, fd); @@ -583,6 +599,16 @@ init_logging(void) pending_cb_messages = smartlist_create(); } +/** Set whether we report logging domains as a part of our log messages. + */ +void +logs_set_domain_logging(int enabled) +{ + LOCK_LOGS(); + log_domains_are_logged = enabled; + UNLOCK_LOGS(); +} + /** Add a log handler to receive messages during startup (before the real * logs are initialized). */ @@ -775,7 +801,6 @@ add_syslog_log(const log_severity_list_t *severity) lf->fd = -1; lf->severities = tor_memdup(severity, sizeof(log_severity_list_t)); lf->filename = tor_strdup(""); - lf->is_syslog = 1; LOCK_LOGS(); @@ -832,18 +857,41 @@ parse_log_domain(const char *domain) } return 0; } -#if 0 -/** Translate a bitmask of log domains to a string, or NULL if the bitmask - * is undecodable. */ -static const char * -domain_to_string(log_domain_mask_t domain) + +/** Translate a bitmask of log domains to a string. */ +static char * +domain_to_string(log_domain_mask_t domain, char *buf, size_t buflen) { - int bit = tor_log2(domain); - if ((bit == 0 && domain == 0) || bit >= N_LOGGING_DOMAINS) - return NULL; - return domain_list[bit]; + char *cp = buf; + char *eos = buf+buflen; + + buf[0] = '\0'; + if (! domain) + return buf; + while (1) { + const char *d; + int bit = tor_log2(domain); + size_t n; + if (bit >= N_LOGGING_DOMAINS) { + tor_snprintf(buf, buflen, "", (long)domain); + return buf+strlen(buf); + } + d = domain_list[bit]; + n = strlcpy(cp, d, eos-cp); + if (n >= buflen) { + tor_snprintf(buf, buflen, "", (long)domain); + return buf+strlen(buf); + } + cp += n; + domain &= ~(1<cfg_ptr. Advance cfg_ptr after * the end of the severityPattern. Set the value of severity_out to diff --git a/src/common/torlog.h b/src/common/torlog.h index f0be73227..791e36394 100644 --- a/src/common/torlog.h +++ b/src/common/torlog.h @@ -132,6 +132,7 @@ int add_file_log(const log_severity_list_t *severity, const char *filename); int add_syslog_log(const log_severity_list_t *severity); #endif int add_callback_log(const log_severity_list_t *severity, log_callback cb); +void logs_set_domain_logging(int enabled); int get_min_log_level(void); void switch_logs_debug(void); void logs_free_all(void); diff --git a/src/or/config.c b/src/or/config.c index 8c1205de4..5600ac516 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -287,6 +287,7 @@ static config_var_t _option_vars[] = { OBSOLETE("IgnoreVersion"), V(KeepalivePeriod, INTERVAL, "5 minutes"), VAR("Log", LINELIST, Logs, NULL), + V(LogMessageDomains, BOOL, "0"), OBSOLETE("LinkPadding"), OBSOLETE("LogLevel"), OBSOLETE("LogFile"), @@ -3811,7 +3812,8 @@ options_transition_affects_workers(or_options_t *old_options, old_options->SafeLogging != new_options->SafeLogging || old_options->ClientOnly != new_options->ClientOnly || public_server_mode(old_options) != public_server_mode(new_options) || - !config_lines_eq(old_options->Logs, new_options->Logs)) + !config_lines_eq(old_options->Logs, new_options->Logs) || + old_options->LogMessageDomains != new_options->LogMessageDomains) return 1; /* Check whether log options match. */ @@ -4387,6 +4389,9 @@ options_init_logs(or_options_t *options, int validate_only) } smartlist_free(elts); + if (ok && !validate_only) + logs_set_domain_logging(options->LogMessageDomains); + return ok?0:-1; } diff --git a/src/or/or.h b/src/or/or.h index 22c8498b6..7bccf8383 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -2338,6 +2338,9 @@ typedef struct { config_line_t *Logs; /**< New-style list of configuration lines * for logs */ + int LogMessageDomains; /**< Boolean: Should we log the domain(s) in which + * each log message occurs? */ + char *DebugLogFile; /**< Where to send verbose log messages. */ char *DataDirectory; /**< OR only: where to store long-term data. */ char *Nickname; /**< OR only: nickname of this onion router. */ From 0ab8b7c0f22bd45d7108ce0185e027cd8e469593 Mon Sep 17 00:00:00 2001 From: Robert Ransom Date: Fri, 4 Feb 2011 05:50:44 -0800 Subject: [PATCH 14/26] Thou shalt not overflow even stupidly small buffers --- src/common/log.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/common/log.c b/src/common/log.c index f58b05b1b..4b21fd916 100644 --- a/src/common/log.c +++ b/src/common/log.c @@ -223,21 +223,31 @@ format_msg(char *buf, size_t buf_len, size_t n; int r; char *end_of_prefix; + char *buf_end; assert(buf_len >= 16); /* prevent integer underflow and general stupidity */ buf_len -= 2; /* subtract 2 characters so we have room for \n\0 */ + buf_end = buf+buf_len; /* point *after* the last char we can write to */ n = _log_prefix(buf, buf_len, severity); end_of_prefix = buf+n; if (log_domains_are_logged) { char *cp = buf+n; + if (cp == buf_end) goto format_msg_no_room_for_domains; *cp++ = '{'; + if (cp == buf_end) goto format_msg_no_room_for_domains; cp = domain_to_string(domain, cp, (buf+buf_len-cp)); + if (cp == buf_end) goto format_msg_no_room_for_domains; *cp++ = '}'; + if (cp == buf_end) goto format_msg_no_room_for_domains; *cp++ = ' '; + if (cp == buf_end) goto format_msg_no_room_for_domains; end_of_prefix = cp; n = cp-buf; + format_msg_no_room_for_domains: + /* This will leave end_of_prefix and n unchanged, and thus cause + * whatever log domain string we had written to be clobbered. */ } if (funcname && should_log_function_name(domain, severity)) { From 5dbaf9dbd545d7c9ecc18224883a74a9dc60ecf4 Mon Sep 17 00:00:00 2001 From: Sebastian Hahn Date: Fri, 11 Feb 2011 16:57:53 +0100 Subject: [PATCH 15/26] Windows has EACCES, not EACCESS Once again spotted by mobmix Also add a changes file for the fix --- changes/bug2504 | 5 +++++ src/common/compat.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) create mode 100644 changes/bug2504 diff --git a/changes/bug2504 b/changes/bug2504 new file mode 100644 index 000000000..791600e59 --- /dev/null +++ b/changes/bug2504 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Fix a bug with our locking implementation on windows that couldn't + correctly detect when a file was already locked. Fixes bug 2504, + bugfix on 0.2.1.6-alpha. + diff --git a/src/common/compat.c b/src/common/compat.c index 5dfde3dc0..d29cacf37 100644 --- a/src/common/compat.c +++ b/src/common/compat.c @@ -699,7 +699,7 @@ tor_lockfile_lock(const char *filename, int blocking, int *locked_out) #ifdef WIN32 _lseek(fd, 0, SEEK_SET); if (_locking(fd, blocking ? _LK_LOCK : _LK_NBLCK, 1) < 0) { - if (errno != EACCESS && errno != EDEADLOCK) + if (errno != EACCES && errno != EDEADLOCK) log_warn(LD_FS,"Couldn't lock \"%s\": %s", filename, strerror(errno)); else *locked_out = 1; From 444a955e4a3877e9a7b3016deee9f1f4dfafdd93 Mon Sep 17 00:00:00 2001 From: Robert Ransom Date: Wed, 16 Feb 2011 04:12:37 -0800 Subject: [PATCH 16/26] Don't crash a bridge authority on SIGHUP if it's not in the consensus Fixes bug 2572. --- changes/bug2572 | 5 +++++ src/or/router.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) create mode 100644 changes/bug2572 diff --git a/changes/bug2572 b/changes/bug2572 new file mode 100644 index 000000000..a5cca284a --- /dev/null +++ b/changes/bug2572 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Don't crash a bridge authority on SIGHUP if it can't force itself + into its routerlist. Fixes bug 2572. + + diff --git a/src/or/router.c b/src/or/router.c index 26ac351fc..59276bac3 100644 --- a/src/or/router.c +++ b/src/or/router.c @@ -629,7 +629,7 @@ init_keys(void) /* 4. Build our router descriptor. */ /* Must be called after keys are initialized. */ mydesc = router_get_my_descriptor(); - if (authdir_mode(options)) { + if (authdir_mode_handles_descs(options, ROUTER_PURPOSE_GENERAL)) { const char *m = NULL; routerinfo_t *ri; /* We need to add our own fingerprint so it gets recognized. */ From 7bdb7d4811bb5ff027e124e6558181167c2e2f91 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Mon, 21 Feb 2011 16:02:16 -0500 Subject: [PATCH 17/26] Remove specs from 0.2.1 branch: they have moved to a new repository. --- doc/Makefile.am | 6 +- doc/spec/Makefile.am | 5 - doc/spec/README | 10 + doc/spec/address-spec.txt | 68 - doc/spec/bridges-spec.txt | 250 --- doc/spec/control-spec-v0.txt | 499 ----- doc/spec/control-spec.txt | 1854 ---------------- doc/spec/dir-spec-v1.txt | 315 --- doc/spec/dir-spec-v2.txt | 897 -------- doc/spec/dir-spec.txt | 1898 ----------------- doc/spec/path-spec.txt | 423 ---- doc/spec/proposals/000-index.txt | 161 -- doc/spec/proposals/001-process.txt | 187 -- doc/spec/proposals/098-todo.txt | 109 - doc/spec/proposals/099-misc.txt | 30 - doc/spec/proposals/100-tor-spec-udp.txt | 424 ---- doc/spec/proposals/101-dir-voting.txt | 285 --- doc/spec/proposals/102-drop-opt.txt | 40 - doc/spec/proposals/103-multilevel-keys.txt | 206 -- doc/spec/proposals/104-short-descriptors.txt | 183 -- doc/spec/proposals/105-handshake-revision.txt | 325 --- .../proposals/106-less-tls-constraint.txt | 113 - .../proposals/107-uptime-sanity-checking.txt | 56 - .../proposals/108-mtbf-based-stability.txt | 90 - doc/spec/proposals/109-no-sharing-ips.txt | 92 - .../proposals/110-avoid-infinite-circuits.txt | 122 -- .../proposals/111-local-traffic-priority.txt | 153 -- .../112-bring-back-pathlencoinweight.txt | 165 -- .../113-fast-authority-interface.txt | 87 - .../proposals/114-distributed-storage.txt | 441 ---- doc/spec/proposals/115-two-hop-paths.txt | 387 ---- .../116-two-hop-paths-from-guard.txt | 120 -- doc/spec/proposals/117-ipv6-exits.txt | 412 ---- doc/spec/proposals/118-multiple-orports.txt | 86 - doc/spec/proposals/119-controlport-auth.txt | 142 -- .../proposals/120-shutdown-descriptors.txt | 85 - .../121-hidden-service-authentication.txt | 778 ------- doc/spec/proposals/122-unnamed-flag.txt | 138 -- doc/spec/proposals/123-autonaming.txt | 56 - doc/spec/proposals/124-tls-certificates.txt | 315 --- doc/spec/proposals/125-bridges.txt | 293 --- doc/spec/proposals/126-geoip-reporting.txt | 412 ---- .../127-dirport-mirrors-downloads.txt | 157 -- doc/spec/proposals/128-bridge-families.txt | 66 - .../proposals/129-reject-plaintext-ports.txt | 116 - doc/spec/proposals/130-v2-conn-protocol.txt | 186 -- doc/spec/proposals/131-verify-tor-usage.txt | 150 -- .../132-browser-check-tor-service.txt | 147 -- doc/spec/proposals/133-unreachable-ors.txt | 128 -- doc/spec/proposals/134-robust-voting.txt | 105 - .../proposals/135-private-tor-networks.txt | 283 --- doc/spec/proposals/136-legacy-keys.txt | 100 - doc/spec/proposals/137-bootstrap-phases.txt | 237 -- ...138-remove-down-routers-from-consensus.txt | 51 - .../139-conditional-consensus-download.txt | 94 - doc/spec/proposals/140-consensus-diffs.txt | 149 -- doc/spec/proposals/141-jit-sd-downloads.txt | 325 --- .../142-combine-intro-and-rend-points.txt | 279 --- .../143-distributed-storage-improvements.txt | 196 -- .../144-enforce-distinct-providers.txt | 165 -- doc/spec/proposals/145-newguard-flag.txt | 41 - .../proposals/146-long-term-stability.txt | 86 - doc/spec/proposals/147-prevoting-opinions.txt | 60 - .../148-uniform-client-end-reason.txt | 59 - doc/spec/proposals/149-using-netinfo-data.txt | 44 - doc/spec/proposals/150-exclude-exit-nodes.txt | 48 - .../151-path-selection-improvements.txt | 147 -- .../proposals/152-single-hop-circuits.txt | 64 - ...153-automatic-software-update-protocol.txt | 177 -- doc/spec/proposals/154-automatic-updates.txt | 379 ---- .../155-four-hidden-service-improvements.txt | 122 -- .../proposals/156-tracking-blocked-ports.txt | 529 ----- .../proposals/157-specific-cert-download.txt | 104 - doc/spec/proposals/158-microdescriptors.txt | 207 -- doc/spec/proposals/159-exit-scanning.txt | 144 -- doc/spec/proposals/ideas/xxx-auto-update.txt | 39 - .../ideas/xxx-bridge-disbursement.txt | 174 -- .../xxx-controllers-intercept-extends.txt | 44 - .../ideas/xxx-exit-scanning-outline.txt | 44 - .../proposals/ideas/xxx-geoip-survey-plan.txt | 137 -- .../ideas/xxx-grand-scaling-plan.txt | 97 - .../proposals/ideas/xxx-hide-platform.txt | 39 - .../proposals/ideas/xxx-port-knocking.txt | 93 - .../proposals/ideas/xxx-rate-limit-exits.txt | 63 - .../ideas/xxx-separate-streams-by-port.txt | 61 - .../proposals/ideas/xxx-what-uses-sha1.txt | 140 -- doc/spec/proposals/reindex.py | 117 - doc/spec/rend-spec.txt | 768 ------- doc/spec/socks-extensions.txt | 79 - doc/spec/tor-spec.txt | 993 --------- doc/spec/version-spec.txt | 45 - 91 files changed, 13 insertions(+), 20783 deletions(-) delete mode 100644 doc/spec/Makefile.am create mode 100644 doc/spec/README delete mode 100644 doc/spec/address-spec.txt delete mode 100644 doc/spec/bridges-spec.txt delete mode 100644 doc/spec/control-spec-v0.txt delete mode 100644 doc/spec/control-spec.txt delete mode 100644 doc/spec/dir-spec-v1.txt delete mode 100644 doc/spec/dir-spec-v2.txt delete mode 100644 doc/spec/dir-spec.txt delete mode 100644 doc/spec/path-spec.txt delete mode 100644 doc/spec/proposals/000-index.txt delete mode 100644 doc/spec/proposals/001-process.txt delete mode 100644 doc/spec/proposals/098-todo.txt delete mode 100644 doc/spec/proposals/099-misc.txt delete mode 100644 doc/spec/proposals/100-tor-spec-udp.txt delete mode 100644 doc/spec/proposals/101-dir-voting.txt delete mode 100644 doc/spec/proposals/102-drop-opt.txt delete mode 100644 doc/spec/proposals/103-multilevel-keys.txt delete mode 100644 doc/spec/proposals/104-short-descriptors.txt delete mode 100644 doc/spec/proposals/105-handshake-revision.txt delete mode 100644 doc/spec/proposals/106-less-tls-constraint.txt delete mode 100644 doc/spec/proposals/107-uptime-sanity-checking.txt delete mode 100644 doc/spec/proposals/108-mtbf-based-stability.txt delete mode 100644 doc/spec/proposals/109-no-sharing-ips.txt delete mode 100644 doc/spec/proposals/110-avoid-infinite-circuits.txt delete mode 100644 doc/spec/proposals/111-local-traffic-priority.txt delete mode 100644 doc/spec/proposals/112-bring-back-pathlencoinweight.txt delete mode 100644 doc/spec/proposals/113-fast-authority-interface.txt delete mode 100644 doc/spec/proposals/114-distributed-storage.txt delete mode 100644 doc/spec/proposals/115-two-hop-paths.txt delete mode 100644 doc/spec/proposals/116-two-hop-paths-from-guard.txt delete mode 100644 doc/spec/proposals/117-ipv6-exits.txt delete mode 100644 doc/spec/proposals/118-multiple-orports.txt delete mode 100644 doc/spec/proposals/119-controlport-auth.txt delete mode 100644 doc/spec/proposals/120-shutdown-descriptors.txt delete mode 100644 doc/spec/proposals/121-hidden-service-authentication.txt delete mode 100644 doc/spec/proposals/122-unnamed-flag.txt delete mode 100644 doc/spec/proposals/123-autonaming.txt delete mode 100644 doc/spec/proposals/124-tls-certificates.txt delete mode 100644 doc/spec/proposals/125-bridges.txt delete mode 100644 doc/spec/proposals/126-geoip-reporting.txt delete mode 100644 doc/spec/proposals/127-dirport-mirrors-downloads.txt delete mode 100644 doc/spec/proposals/128-bridge-families.txt delete mode 100644 doc/spec/proposals/129-reject-plaintext-ports.txt delete mode 100644 doc/spec/proposals/130-v2-conn-protocol.txt delete mode 100644 doc/spec/proposals/131-verify-tor-usage.txt delete mode 100644 doc/spec/proposals/132-browser-check-tor-service.txt delete mode 100644 doc/spec/proposals/133-unreachable-ors.txt delete mode 100644 doc/spec/proposals/134-robust-voting.txt delete mode 100644 doc/spec/proposals/135-private-tor-networks.txt delete mode 100644 doc/spec/proposals/136-legacy-keys.txt delete mode 100644 doc/spec/proposals/137-bootstrap-phases.txt delete mode 100644 doc/spec/proposals/138-remove-down-routers-from-consensus.txt delete mode 100644 doc/spec/proposals/139-conditional-consensus-download.txt delete mode 100644 doc/spec/proposals/140-consensus-diffs.txt delete mode 100644 doc/spec/proposals/141-jit-sd-downloads.txt delete mode 100644 doc/spec/proposals/142-combine-intro-and-rend-points.txt delete mode 100644 doc/spec/proposals/143-distributed-storage-improvements.txt delete mode 100644 doc/spec/proposals/144-enforce-distinct-providers.txt delete mode 100644 doc/spec/proposals/145-newguard-flag.txt delete mode 100644 doc/spec/proposals/146-long-term-stability.txt delete mode 100644 doc/spec/proposals/147-prevoting-opinions.txt delete mode 100644 doc/spec/proposals/148-uniform-client-end-reason.txt delete mode 100644 doc/spec/proposals/149-using-netinfo-data.txt delete mode 100644 doc/spec/proposals/150-exclude-exit-nodes.txt delete mode 100644 doc/spec/proposals/151-path-selection-improvements.txt delete mode 100644 doc/spec/proposals/152-single-hop-circuits.txt delete mode 100644 doc/spec/proposals/153-automatic-software-update-protocol.txt delete mode 100644 doc/spec/proposals/154-automatic-updates.txt delete mode 100644 doc/spec/proposals/155-four-hidden-service-improvements.txt delete mode 100644 doc/spec/proposals/156-tracking-blocked-ports.txt delete mode 100644 doc/spec/proposals/157-specific-cert-download.txt delete mode 100644 doc/spec/proposals/158-microdescriptors.txt delete mode 100644 doc/spec/proposals/159-exit-scanning.txt delete mode 100644 doc/spec/proposals/ideas/xxx-auto-update.txt delete mode 100644 doc/spec/proposals/ideas/xxx-bridge-disbursement.txt delete mode 100644 doc/spec/proposals/ideas/xxx-controllers-intercept-extends.txt delete mode 100644 doc/spec/proposals/ideas/xxx-exit-scanning-outline.txt delete mode 100644 doc/spec/proposals/ideas/xxx-geoip-survey-plan.txt delete mode 100644 doc/spec/proposals/ideas/xxx-grand-scaling-plan.txt delete mode 100644 doc/spec/proposals/ideas/xxx-hide-platform.txt delete mode 100644 doc/spec/proposals/ideas/xxx-port-knocking.txt delete mode 100644 doc/spec/proposals/ideas/xxx-rate-limit-exits.txt delete mode 100644 doc/spec/proposals/ideas/xxx-separate-streams-by-port.txt delete mode 100644 doc/spec/proposals/ideas/xxx-what-uses-sha1.txt delete mode 100755 doc/spec/proposals/reindex.py delete mode 100644 doc/spec/rend-spec.txt delete mode 100644 doc/spec/socks-extensions.txt delete mode 100644 doc/spec/tor-spec.txt delete mode 100644 doc/spec/version-spec.txt diff --git a/doc/Makefile.am b/doc/Makefile.am index 79123d81d..295981375 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -2,11 +2,11 @@ EXTRA_DIST = HACKING \ tor-resolve.1 tor-gencert.1 \ tor-osx-dmg-creation.txt tor-rpm-creation.txt \ - tor-win32-mingw-creation.txt + tor-win32-mingw-creation.txt spec/README man_MANS = tor.1 tor-resolve.1 tor-gencert.1 -SUBDIRS = design-paper spec +SUBDIRS = design-paper -DIST_SUBDIRS = design-paper spec +DIST_SUBDIRS = design-paper diff --git a/doc/spec/Makefile.am b/doc/spec/Makefile.am deleted file mode 100644 index 208901d9d..000000000 --- a/doc/spec/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ - -EXTRA_DIST = tor-spec.txt rend-spec.txt control-spec.txt \ - dir-spec.txt socks-extensions.txt path-spec.txt \ - version-spec.txt address-spec.txt - diff --git a/doc/spec/README b/doc/spec/README new file mode 100644 index 000000000..a7fa17002 --- /dev/null +++ b/doc/spec/README @@ -0,0 +1,10 @@ +The Tor specifications and proposals have moved to a new repository. + +To browse the specifications, go to + https://gitweb.torproject.org/torspec.git/tree + +To check out the specification repository, run + git clone git://git.torproject.org/torspec.git + +For other information on the repository, see + http://gitweb.torproject.org/torspec.git diff --git a/doc/spec/address-spec.txt b/doc/spec/address-spec.txt deleted file mode 100644 index 2a84d857e..000000000 --- a/doc/spec/address-spec.txt +++ /dev/null @@ -1,68 +0,0 @@ -$Id$ - - Special Hostnames in Tor - Nick Mathewson - -1. Overview - - Most of the time, Tor treats user-specified hostnames as opaque: When - the user connects to www.torproject.org, Tor picks an exit node and uses - that node to connect to "www.torproject.org". Some hostnames, however, - can be used to override Tor's default behavior and circuit-building - rules. - - These hostnames can be passed to Tor as the address part of a SOCKS4a or - SOCKS5 request. If the application is connected to Tor using an IP-only - method (such as SOCKS4, TransPort, or NatdPort), these hostnames can be - substituted for certain IP addresses using the MapAddress configuration - option or the MAPADDRESS control command. - -2. .exit - - SYNTAX: [hostname].[name-or-digest].exit - [name-or-digest].exit - - Hostname is a valid hostname; [name-or-digest] is either the nickname of a - Tor node or the hex-encoded digest of that node's public key. - - When Tor sees an address in this format, it uses the specified hostname as - the exit node. If no "hostname" component is given, Tor defaults to the - published IPv4 address of the exit node. - - It is valid to try to resolve hostnames, and in fact upon success Tor - will cache an internal mapaddress of the form - "www.google.com.foo.exit=64.233.161.99.foo.exit" to speed subsequent - lookups. - - EXAMPLES: - www.example.com.exampletornode.exit - - Connect to www.example.com from the node called "exampletornode." - - exampletornode.exit - - Connect to the published IP address of "exampletornode" using - "exampletornode" as the exit. - -3. .onion - - SYNTAX: [digest].onion - - The digest is the first eighty bits of a SHA1 hash of the identity key for - a hidden service, encoded in base32. - - When Tor sees an address in this format, it tries to look up and connect to - the specified hidden service. See rend-spec.txt for full details. - -4. .noconnect - - SYNTAX: [string].noconnect - - When Tor sees an address in this format, it immediately closes the - connection without attaching it to any circuit. This is useful for - controllers that want to test whether a given application is indeed using - the same instance of Tor that they're controlling. - -5. [XXX Is there a ".virtual" address that we expose too, or is that -just intended to be internal? -RD] - diff --git a/doc/spec/bridges-spec.txt b/doc/spec/bridges-spec.txt deleted file mode 100644 index 4a9b373c8..000000000 --- a/doc/spec/bridges-spec.txt +++ /dev/null @@ -1,250 +0,0 @@ -$Id$ - - Tor bridges specification - -0. Preface - - This document describes the design decisions around support for bridge - users, bridge relays, and bridge authorities. It acts as an overview - of the bridge design and deployment for developers, and it also tries - to point out limitations in the current design and implementation. - - For more details on what all of these mean, look at blocking.tex in - /doc/design-paper/ - -1. Bridge relays - - Bridge relays are just like normal Tor relays except they don't publish - their server descriptors to the main directory authorities. - -1.1. PublishServerDescriptor - - To configure your relay to be a bridge relay, just add - BridgeRelay 1 - PublishServerDescriptor bridge - to your torrc. This will cause your relay to publish its descriptor - to the bridge authorities rather than to the default authorities. - - Alternatively, you can say - BridgeRelay 1 - PublishServerDescriptor 0 - which will cause your relay to not publish anywhere. This could be - useful for private bridges. - -1.2. Recommendations. - - Bridge relays should use an exit policy of "reject *:*". This is - because they only need to relay traffic between the bridge users - and the rest of the Tor network, so there's no need to let people - exit directly from them. - - We invented the RelayBandwidth* options for this situation: Tor clients - who want to allow relaying too. See proposal 111 for details. Relay - operators should feel free to rate-limit their relayed traffic. - -1.3. Implementation note. - - Vidalia 0.0.15 has turned its "Relay" settings page into a tri-state - "Don't relay" / "Relay for the Tor network" / "Help censored users". - - If you click the third choice, it forces your exit policy to reject *:*. - - If all the bridges end up on port 9001, that's not so good. On the - other hand, putting the bridges on a low-numbered port in the Unix - world requires jumping through extra hoops. The current compromise is - that Vidalia makes the ORPort default to 443 on Windows, and 9001 on - other platforms. - - At the bottom of the relay config settings window, Vidalia displays - the bridge identifier to the operator (see Section 3.1) so he can pass - it on to bridge users. - -2. Bridge authorities. - - Bridge authorities are like normal v3 directory authorities, except - they don't create their own network-status documents or votes. So if - you ask a bridge authority for a network-status document or consensus, - they behave like a directory mirror: they give you one from one of - the main authorities. But if you ask the bridge authority for the - descriptor corresponding to a particular identity fingerprint, it will - happily give you the latest descriptor for that fingerprint. - - To become a bridge authority, add these lines to your torrc: - AuthoritativeDirectory 1 - BridgeAuthoritativeDir 1 - - Right now there's one bridge authority, running on the Tonga relay. - -2.1. Exporting bridge-purpose descriptors - - We've added a new purpose for server descriptors: the "bridge" - purpose. With the new router-descriptors file format that includes - annotations, it's easy to look through it and find the bridge-purpose - descriptors. - - Currently we export the bridge descriptors from Tonga to the - BridgeDB server, so it can give them out according to the policies - in blocking.pdf. - -2.2. Reachability/uptime testing - - Right now the bridge authorities do active reachability testing of - bridges, so we know which ones to recommend for users. - - But in the design document, we suggested that bridges should publish - anonymously (i.e. via Tor) to the bridge authority, so somebody watching - the bridge authority can't just enumerate all the bridges. But if we're - doing active measurement, the game is up. Perhaps we should back off on - this goal, or perhaps we should do our active measurement anonymously? - - Answering this issue is scheduled for 0.2.1.x. - -2.3. Future work: migrating to multiple bridge authorities - - Having only one bridge authority is both a trust bottleneck (if you - break into one place you learn about every single bridge we've got) - and a robustness bottleneck (when it's down, bridge users become sad). - - Right now if we put up a second bridge authority, all the bridges would - publish to it, and (assuming the code works) bridge users would query - a random bridge authority. This resolves the robustness bottleneck, - but makes the trust bottleneck even worse. - - In 0.2.2.x and later we should think about better ways to have multiple - bridge authorities. - -3. Bridge users. - - Bridge users are like ordinary Tor users except they use encrypted - directory connections by default, and they use bridge relays as both - entry guards (their first hop) and directory guards (the source of - all their directory information). - - To become a bridge user, add the following line to your torrc: - UseBridges 1 - - and then add at least one "Bridge" line to your torrc based on the - format below. - -3.1. Format of the bridge identifier. - - The canonical format for a bridge identifier contains an IP address, - an ORPort, and an identity fingerprint: - bridge 128.31.0.34:9009 4C17 FB53 2E20 B2A8 AC19 9441 ECD2 B017 7B39 E4B1 - - However, the identity fingerprint can be left out, in which case the - bridge user will connect to that relay and use it as a bridge regardless - of what identity key it presents: - bridge 128.31.0.34:9009 - This might be useful for cases where only short bridge identifiers - can be communicated to bridge users. - - In a future version we may also support bridge identifiers that are - only a key fingerprint: - bridge 4C17 FB53 2E20 B2A8 AC19 9441 ECD2 B017 7B39 E4B1 - and the bridge user can fetch the latest descriptor from the bridge - authority (see Section 3.4). - -3.2. Bridges as entry guards - - For now, bridge users add their bridge relays to their list of "entry - guards" (see path-spec.txt for background on entry guards). They are - managed by the entry guard algorithms exactly as if they were a normal - entry guard -- their keys and timing get cached in the "state" file, - etc. This means that when the Tor user starts up with "UseBridges" - disabled, he will skip past the bridge entries since they won't be - listed as up and usable in his networkstatus consensus. But to be clear, - the "entry_guards" list doesn't currently distinguish guards by purpose. - - Internally, each bridge user keeps a smartlist of "bridge_info_t" - that reflects the "bridge" lines from his torrc along with a download - schedule (see Section 3.5 below). When he starts Tor, he attempts - to fetch a descriptor for each configured bridge (see Section 3.4 - below). When he succeeds at getting a descriptor for one of the bridges - in his list, he adds it directly to the entry guard list using the - normal add_an_entry_guard() interface. Once a bridge descriptor has - been added, should_delay_dir_fetches() will stop delaying further - directory fetches, and the user begins to bootstrap his directory - information from that bridge (see Section 3.3). - - Currently bridge users cache their bridge descriptors to the - "cached-descriptors" file (annotated with purpose "bridge"), but - they don't make any attempt to reuse descriptors they find in this - file. The theory is that either the bridge is available now, in which - case you can get a fresh descriptor, or it's not, in which case an - old descriptor won't do you much good. - - We could disable writing out the bridge lines to the state file, if - we think this is a problem. - - As an exception, if we get an application request when we have one - or more bridge descriptors but we believe none of them are running, - we mark them all as running again. This is similar to the exception - already in place to help long-idle Tor clients realize they should - fetch fresh directory information rather than just refuse requests. - -3.3. Bridges as directory guards - - In addition to using bridges as the first hop in their circuits, bridge - users also use them to fetch directory updates. Other than initial - bootstrapping to find a working bridge descriptor (see Section 3.4 - below), all further non-anonymized directory fetches will be redirected - to the bridge. - - This means that bridge relays need to have cached answers for all - questions the bridge user might ask. This makes the upgrade path - tricky --- for example, if we migrate to a v4 directory design, the - bridge user would need to keep using v3 so long as his bridge relays - only knew how to answer v3 queries. - - In a future design, for cases where the user has enough information - to build circuits yet the chosen bridge doesn't know how to answer a - given query, we might teach bridge users to make an anonymized request - to a more suitable directory server. - -3.4. How bridge users get their bridge descriptor - - Bridge users can fetch bridge descriptors in two ways: by going directly - to the bridge and asking for "/tor/server/authority", or by going to - the bridge authority and asking for "/tor/server/fp/ID". By default, - they will only try the direct queries. If the user sets - UpdateBridgesFromAuthority 1 - in his config file, then he will try querying the bridge authority - first for bridges where he knows a digest (if he only knows an IP - address and ORPort, then his only option is a direct query). - - If the user has at least one working bridge, then he will do further - queries to the bridge authority through a full three-hop Tor circuit. - But when bootstrapping, he will make a direct begin_dir-style connection - to the bridge authority. - - As of Tor 0.2.0.10-alpha, if the user attempts to fetch a descriptor - from the bridge authority and it returns a 404 not found, the user - will automatically fall back to trying a direct query. Therefore it is - recommended that bridge users always set UpdateBridgesFromAuthority, - since at worst it will delay their fetches a little bit and notify - the bridge authority of the identity fingerprint (but not location) - of their intended bridges. - -3.5. Bridge descriptor retry schedule - - Bridge users try to fetch a descriptor for each bridge (using the - steps in Section 3.4 above) on startup. Whenever they receive a - bridge descriptor, they reschedule a new descriptor download for 1 - hour from then. - - If on the other hand it fails, they try again after 15 minutes for the - first attempt, after 15 minutes for the second attempt, and after 60 - minutes for subsequent attempts. - - In 0.2.2.x we should come up with some smarter retry schedules. - -3.6. Implementation note. - - Vidalia 0.1.0 has a new checkbox in its Network config window called - "My ISP blocks connections to the Tor network." Users who click that - box change their configuration to: - UseBridges 1 - UpdateBridgesFromAuthority 1 - and should add at least one bridge identifier. - diff --git a/doc/spec/control-spec-v0.txt b/doc/spec/control-spec-v0.txt deleted file mode 100644 index faf75a64a..000000000 --- a/doc/spec/control-spec-v0.txt +++ /dev/null @@ -1,499 +0,0 @@ -$Id$ - - TC: A Tor control protocol (Version 0) - --1. Deprecation - -THIS PROTOCOL IS DEPRECATED. It is still documented here because Tor -0.1.1.x happens to support much of it; but the support for v0 is not -maintained, so you should expect it to rot in unpredictable ways. Support -for v0 will be removed some time after Tor 0.1.2. - -0. Scope - -This document describes an implementation-specific protocol that is used -for other programs (such as frontend user-interfaces) to communicate -with a locally running Tor process. It is not part of the Tor onion -routing protocol. - -We're trying to be pretty extensible here, but not infinitely -forward-compatible. - -1. Protocol outline - -TC is a bidirectional message-based protocol. It assumes an underlying -stream for communication between a controlling process (the "client") and -a Tor process (the "server"). The stream may be implemented via TCP, -TLS-over-TCP, a Unix-domain socket, or so on, but it must provide -reliable in-order delivery. For security, the stream should not be -accessible by untrusted parties. - -In TC, the client and server send typed variable-length messages to each -other over the underlying stream. By default, all messages from the server -are in response to messages from the client. Some client requests, however, -will cause the server to send messages to the client indefinitely far into -the future. - -Servers respond to messages in the order they're received. - -2. Message format - -The messages take the following format: - - Length [2 octets; big-endian] - Type [2 octets; big-endian] - Body [Length octets] - -Upon encountering a recognized Type, implementations behave as described in -section 3 below. If the type is not recognized, servers respond with an -"ERROR" message (code UNRECOGNIZED; see 3.1 below), and clients simply ignore -the message. - -2.1. Types and encodings - - All numbers are given in big-endian (network) order. - - OR identities are given in hexadecimal, in the same format as identity key - fingerprints, but without spaces; see tor-spec.txt for more information. - -3. Message types - - Message types are drawn from the following ranges: - - 0x0000-0xEFFF : Reserved for use by official versions of this spec. - 0xF000-0xFFFF : Unallocated; usable by unofficial extensions. - -3.1. ERROR (Type 0x0000) - - Sent in response to a message that could not be processed as requested. - - The body of the message begins with a 2-byte error code. The following - values are defined: - - 0x0000 Unspecified error - [] - - 0x0001 Internal error - [Something went wrong inside Tor, so that the client's - request couldn't be fulfilled.] - - 0x0002 Unrecognized message type - [The client sent a message type we don't understand.] - - 0x0003 Syntax error - [The client sent a message body in a format we can't parse.] - - 0x0004 Unrecognized configuration key - [The client tried to get or set a configuration option we don't - recognize.] - - 0x0005 Invalid configuration value - [The client tried to set a configuration option to an - incorrect, ill-formed, or impossible value.] - - 0x0006 Unrecognized byte code - [The client tried to set a byte code (in the body) that - we don't recognize.] - - 0x0007 Unauthorized. - [The client tried to send a command that requires - authorization, but it hasn't sent a valid AUTHENTICATE - message.] - - 0x0008 Failed authentication attempt - [The client sent a well-formed authorization message.] - - 0x0009 Resource exhausted - [The server didn't have enough of a given resource to - fulfill a given request.] - - 0x000A No such stream - - 0x000B No such circuit - - 0x000C No such OR - - The rest of the body should be a human-readable description of the error. - - In general, new error codes should only be added when they don't fall under - one of the existing error codes. - -3.2. DONE (Type 0x0001) - - Sent from server to client in response to a request that was successfully - completed, with no more information needed. The body is usually empty but - may contain a message. - -3.3. SETCONF (Type 0x0002) - - Change the value of a configuration variable. The body contains a list of - newline-terminated key-value configuration lines. An individual key-value - configuration line consists of the key, followed by a space, followed by - the value. The server behaves as though it had just read the key-value pair - in its configuration file. - - The server responds with a DONE message on success, or an ERROR message on - failure. - - When a configuration options takes multiple values, or when multiple - configuration keys form a context-sensitive group (see below), then - setting _any_ of the options in a SETCONF command is taken to reset all of - the others. For example, if two ORBindAddress values are configured, - and a SETCONF command arrives containing a single ORBindAddress value, the - new command's value replaces the two old values. - - To _remove_ all settings for a given option entirely (and go back to its - default value), send a single line containing the key and no value. - -3.4. GETCONF (Type 0x0003) - - Request the value of a configuration variable. The body contains one or - more NL-terminated strings for configuration keys. The server replies - with a CONFVALUE message. - - If an option appears multiple times in the configuration, all of its - key-value pairs are returned in order. - - Some options are context-sensitive, and depend on other options with - different keywords. These cannot be fetched directly. Currently there - is only one such option: clients should use the "HiddenServiceOptions" - virtual keyword to get all HiddenServiceDir, HiddenServicePort, - HiddenServiceNodes, and HiddenServiceExcludeNodes option settings. - -3.5. CONFVALUE (Type 0x0004) - - Sent in response to a GETCONF message; contains a list of "Key Value\n" - (A non-whitespace keyword, a single space, a non-NL value, a NL) - strings. - -3.6. SETEVENTS (Type 0x0005) - - Request the server to inform the client about interesting events. - The body contains a list of 2-byte event codes (see "event" below). - Any events *not* listed in the SETEVENTS body are turned off; thus, sending - SETEVENTS with an empty body turns off all event reporting. - - The server responds with a DONE message on success, and an ERROR message - if one of the event codes isn't recognized. (On error, the list of active - event codes isn't changed.) - -3.7. EVENT (Type 0x0006) - - Sent from the server to the client when an event has occurred and the - client has requested that kind of event. The body contains a 2-byte - event code followed by additional event-dependent information. Event - codes are: - 0x0001 -- Circuit status changed - - Status [1 octet] - 0x00 Launched - circuit ID assigned to new circuit - 0x01 Built - all hops finished, can now accept streams - 0x02 Extended - one more hop has been completed - 0x03 Failed - circuit closed (was not built) - 0x04 Closed - circuit closed (was built) - Circuit ID [4 octets] - (Must be unique to Tor process/time) - Path [NUL-terminated comma-separated string] - (For extended/failed, is the portion of the path that is - built) - - 0x0002 -- Stream status changed - - Status [1 octet] - (Sent connect=0,sent resolve=1,succeeded=2,failed=3, - closed=4, new connection=5, new resolve request=6, - stream detached from circuit and still retriable=7) - Stream ID [4 octets] - (Must be unique to Tor process/time) - Target (NUL-terminated address-port string] - - 0x0003 -- OR Connection status changed - - Status [1 octet] - (Launched=0,connected=1,failed=2,closed=3) - OR nickname/identity [NUL-terminated] - - 0x0004 -- Bandwidth used in the last second - - Bytes read [4 octets] - Bytes written [4 octets] - - 0x0005 -- Notice/warning/error occurred - - Message [NUL-terminated] - - - - 0x0006 -- New descriptors available - - OR List [NUL-terminated, comma-delimited list of - OR identity] - - 0x0007 -- Debug message occurred - 0x0008 -- Info message occurred - 0x0009 -- Notice message occurred - 0x000A -- Warning message occurred - 0x000B -- Error message occurred - - Message [NUL-terminated] - -3.8. AUTHENTICATE (Type 0x0007) - - Sent from the client to the server. Contains a 'magic cookie' to prove - that client is really allowed to control this Tor process. The server - responds with DONE or ERROR. - - The format of the 'cookie' is implementation-dependent; see 4.1 below for - information on how the standard Tor implementation handles it. - -3.9. SAVECONF (Type 0x0008) - - Sent from the client to the server. Instructs the server to write out - its config options into its torrc. Server returns DONE if successful, or - ERROR if it can't write the file or some other error occurs. - -3.10. SIGNAL (Type 0x0009) - - Sent from the client to the server. The body contains one byte that - indicates the action the client wishes the server to take. - - 1 (0x01) -- Reload: reload config items, refetch directory. - 2 (0x02) -- Controlled shutdown: if server is an OP, exit immediately. - If it's an OR, close listeners and exit after 30 seconds. - 10 (0x0A) -- Dump stats: log information about open connections and - circuits. - 12 (0x0C) -- Debug: switch all open logs to loglevel debug. - 15 (0x0F) -- Immediate shutdown: clean up and exit now. - - The server responds with DONE if the signal is recognized (or simply - closes the socket if it was asked to close immediately), else ERROR. - -3.11. MAPADDRESS (Type 0x000A) - - Sent from the client to the server. The body contains a sequence of - address mappings, each consisting of the address to be mapped, a single - space, the replacement address, and a NL character. - - Addresses may be IPv4 addresses, IPv6 addresses, or hostnames. - - The client sends this message to the server in order to tell it that future - SOCKS requests for connections to the original address should be replaced - with connections to the specified replacement address. If the addresses - are well-formed, and the server is able to fulfill the request, the server - replies with a single DONE message containing the source and destination - addresses. If request is malformed, the server replies with a syntax error - message. The server can't fulfill the request, it replies with an internal - ERROR message. - - The client may decline to provide a body for the original address, and - instead send a special null address ("0.0.0.0" for IPv4, "::0" for IPv6, or - "." for hostname), signifying that the server should choose the original - address itself, and return that address in the DONE message. The server - should ensure that it returns an element of address space that is unlikely - to be in actual use. If there is already an address mapped to the - destination address, the server may reuse that mapping. - - If the original address is already mapped to a different address, the old - mapping is removed. If the original address and the destination address - are the same, the server removes any mapping in place for the original - address. - - {Note: This feature is designed to be used to help Tor-ify applications - that need to use SOCKS4 or hostname-less SOCKS5. There are three - approaches to doing this: - 1. Somehow make them use SOCKS4a or SOCKS5-with-hostnames instead. - 2. Use tor-resolve (or another interface to Tor's resolve-over-SOCKS - feature) to resolve the hostname remotely. This doesn't work - with special addresses like x.onion or x.y.exit. - 3. Use MAPADDRESS to map an IP address to the desired hostname, and then - arrange to fool the application into thinking that the hostname - has resolved to that IP. - This functionality is designed to help implement the 3rd approach.} - - [XXXX When, if ever, can mappings expire? Should they expire?] - [XXXX What addresses, if any, are safe to use?] - -3.12 GETINFO (Type 0x000B) - - Sent from the client to the server. The message body is as for GETCONF: - one or more NL-terminated strings. The server replies with an INFOVALUE - message. - - Unlike GETCONF, this message is used for data that are not stored in the - Tor configuration file, but instead. - - Recognized key and their values include: - - "version" -- The version of the server's software, including the name - of the software. (example: "Tor 0.0.9.4") - - "desc/id/" or "desc/name/" -- the latest server - descriptor for a given OR, NUL-terminated. If no such OR is known, the - corresponding value is an empty string. - - "network-status" -- a space-separated list of all known OR identities. - This is in the same format as the router-status line in directories; - see tor-spec.txt for details. - - "addr-mappings/all" - "addr-mappings/config" - "addr-mappings/cache" - "addr-mappings/control" -- a NL-terminated list of address mappings, each - in the form of "from-address" SP "to-address". The 'config' key - returns those address mappings set in the configuration; the 'cache' - key returns the mappings in the client-side DNS cache; the 'control' - key returns the mappings set via the control interface; the 'all' - target returns the mappings set through any mechanism. - -3.13 INFOVALUE (Type 0x000C) - - Sent from the server to the client in response to a GETINFO message. - Contains one or more items of the format: - - Key [(NUL-terminated string)] - Value [(NUL-terminated string)] - - The keys match those given in the GETINFO message. - -3.14 EXTENDCIRCUIT (Type 0x000D) - - Sent from the client to the server. The message body contains two fields: - Circuit ID [4 octets] - Path [NUL-terminated, comma-delimited string of OR nickname/identity] - - This request takes one of two forms: either the Circuit ID is zero, in - which case it is a request for the server to build a new circuit according - to the specified path, or the Circuit ID is nonzero, in which case it is a - request for the server to extend an existing circuit with that ID according - to the specified path. - - If the request is successful, the server sends a DONE message containing - a message body consisting of the four-octet Circuit ID of the newly created - circuit. - -3.15 ATTACHSTREAM (Type 0x000E) - - Sent from the client to the server. The message body contains two fields: - Stream ID [4 octets] - Circuit ID [4 octets] - - This message informs the server that the specified stream should be - associated with the specified circuit. Each stream may be associated with - at most one circuit, and multiple streams may share the same circuit. - Streams can only be attached to completed circuits (that is, circuits that - have sent a circuit status 'built' event). - - If the circuit ID is 0, responsibility for attaching the given stream is - returned to Tor. - - {Implementation note: By default, Tor automatically attaches streams to - circuits itself, unless the configuration variable - "__LeaveStreamsUnattached" is set to "1". Attempting to attach streams - via TC when "__LeaveStreamsUnattached" is false may cause a race between - Tor and the controller, as both attempt to attach streams to circuits.} - -3.16 POSTDESCRIPTOR (Type 0x000F) - - Sent from the client to the server. The message body contains one field: - Descriptor [NUL-terminated string] - - This message informs the server about a new descriptor. - - The descriptor, when parsed, must contain a number of well-specified - fields, including fields for its nickname and identity. - - If there is an error in parsing the descriptor, the server must send an - appropriate error message. If the descriptor is well-formed but the server - chooses not to add it, it must reply with a DONE message whose body - explains why the server was not added. - -3.17 FRAGMENTHEADER (Type 0x0010) - - Sent in either direction. Used to encapsulate messages longer than 65535 - bytes in length. - - Underlying type [2 bytes] - Total Length [4 bytes] - Data [Rest of message] - - A FRAGMENTHEADER message MUST be followed immediately by a number of - FRAGMENT messages, such that lengths of the "Data" fields of the - FRAGMENTHEADER and FRAGMENT messages add to the "Total Length" field of the - FRAGMENTHEADER message. - - Implementations MUST NOT fragment messages of length less than 65536 bytes. - Implementations MUST be able to process fragmented messages that not - optimally packed. - -3.18 FRAGMENT (Type 0x0011) - - Data [Entire message] - - See FRAGMENTHEADER for more information - -3.19 REDIRECTSTREAM (Type 0x0012) - - Sent from the client to the server. The message body contains two fields: - Stream ID [4 octets] - Address [variable-length, NUL-terminated.] - - Tells the server to change the exit address on the specified stream. No - remapping is performed on the new provided address. - - To be sure that the modified address will be used, this event must be sent - after a new stream event is received, and before attaching this stream to - a circuit. - -3.20 CLOSESTREAM (Type 0x0013) - - Sent from the client to the server. The message body contains three - fields: - Stream ID [4 octets] - Reason [1 octet] - Flags [1 octet] - - Tells the server to close the specified stream. The reason should be - one of the Tor RELAY_END reasons given in tor-spec.txt. Flags is not - used currently. Tor may hold the stream open for a while to flush - any data that is pending. - -3.21 CLOSECIRCUIT (Type 0x0014) - - Sent from the client to the server. The message body contains two - fields: - Circuit ID [4 octets] - Flags [1 octet] - - Tells the server to close the specified circuit. If the LSB of the flags - field is nonzero, do not close the circuit unless it is unused. - -4. Implementation notes - -4.1. Authentication - - By default, the current Tor implementation trusts all local users. - - If the 'CookieAuthentication' option is true, Tor writes a "magic cookie" - file named "control_auth_cookie" into its data directory. To authenticate, - the controller must send the contents of this file. - - If the 'HashedControlPassword' option is set, it must contain the salted - hash of a secret password. The salted hash is computed according to the - S2K algorithm in RFC 2440 (OpenPGP), and prefixed with the s2k specifier. - This is then encoded in hexadecimal, prefixed by the indicator sequence - "16:". Thus, for example, the password 'foo' could encode to: - 16:660537E3E1CD49996044A3BF558097A981F539FEA2F9DA662B4626C1C2 - ++++++++++++++++**^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - salt hashed value - indicator - You can generate the salt of a password by calling - 'tor --hash-password ' - or by using the example code in the Python and Java controller libraries. - To authenticate under this scheme, the controller sends Tor the original - secret that was used to generate the password. - -4.2. Don't let the buffer get too big. - - If you ask for lots of events, and 16MB of them queue up on the buffer, - the Tor process will close the socket. - diff --git a/doc/spec/control-spec.txt b/doc/spec/control-spec.txt deleted file mode 100644 index cf92e2b9e..000000000 --- a/doc/spec/control-spec.txt +++ /dev/null @@ -1,1854 +0,0 @@ -$Id$ - - TC: A Tor control protocol (Version 1) - -0. Scope - - This document describes an implementation-specific protocol that is used - for other programs (such as frontend user-interfaces) to communicate with a - locally running Tor process. It is not part of the Tor onion routing - protocol. - - This protocol replaces version 0 of TC, which is now deprecated. For - reference, TC is described in "control-spec-v0.txt". Implementors are - recommended to avoid using TC directly, but instead to use a library that - can easily be updated to use the newer protocol. (Version 0 is used by Tor - versions 0.1.0.x; the protocol in this document only works with Tor - versions in the 0.1.1.x series and later.) - -1. Protocol outline - - TC is a bidirectional message-based protocol. It assumes an underlying - stream for communication between a controlling process (the "client" - or "controller") and a Tor process (or "server"). The stream may be - implemented via TCP, TLS-over-TCP, a Unix-domain socket, or so on, - but it must provide reliable in-order delivery. For security, the - stream should not be accessible by untrusted parties. - - In TC, the client and server send typed messages to each other over the - underlying stream. The client sends "commands" and the server sends - "replies". - - By default, all messages from the server are in response to messages from - the client. Some client requests, however, will cause the server to send - messages to the client indefinitely far into the future. Such - "asynchronous" replies are marked as such. - - Servers respond to messages in the order messages are received. - -2. Message format - -2.1. Description format - - The message formats listed below use ABNF as described in RFC 2234. - The protocol itself is loosely based on SMTP (see RFC 2821). - - We use the following nonterminals from RFC 2822: atom, qcontent - - We define the following general-use nonterminals: - - String = DQUOTE *qcontent DQUOTE - - There are explicitly no limits on line length. All 8-bit characters are - permitted unless explicitly disallowed. - - Wherever CRLF is specified to be accepted from the controller, Tor MAY also - accept LF. Tor, however, MUST NOT generate LF instead of CRLF. - Controllers SHOULD always send CRLF. - -2.2. Commands from controller to Tor - - Command = Keyword Arguments CRLF / "+" Keyword Arguments CRLF Data - Keyword = 1*ALPHA - Arguments = *(SP / VCHAR) - - Specific commands and their arguments are described below in section 3. - -2.3. Replies from Tor to the controller - - Reply = SyncReply / AsyncReply - SyncReply = *(MidReplyLine / DataReplyLine) EndReplyLine - AsyncReply = *(MidReplyLine / DataReplyLine) EndReplyLine - - MidReplyLine = StatusCode "-" ReplyLine - DataReplyLine = StatusCode "+" ReplyLine Data - EndReplyLine = StatusCode SP ReplyLine - ReplyLine = [ReplyText] CRLF - ReplyText = XXXX - StatusCode = 3DIGIT - - Specific replies are mentioned below in section 3, and described more fully - in section 4. - - [Compatibility note: versions of Tor before 0.2.0.3-alpha sometimes - generate AsyncReplies of the form "*(MidReplyLine / DataReplyLine)". - This is incorrect, but controllers that need to work with these - versions of Tor should be prepared to get multi-line AsyncReplies with - the final line (usually "650 OK") omitted.] - -2.4. General-use tokens - - ; Identifiers for servers. - ServerID = Nickname / Fingerprint - - Nickname = 1*19 NicknameChar - NicknameChar = "a"-"z" / "A"-"Z" / "0" - "9" - Fingerprint = "$" 40*HEXDIG - - ; A "=" indicates that the given nickname is canonical; a "~" indicates - ; that the given nickname is not canonical. If no nickname is given at - ; all, Tor does not even have a guess for what this router calls itself. - LongName = Fingerprint [ ( "=" / "~" ) Nickname ] - - ; How a controller tells Tor about a particular OR. There are four - ; possible formats: - ; $Digest -- The router whose identity key hashes to the given digest. - ; This is the preferred way to refer to an OR. - ; $Digest~Name -- The router whose identity key hashes to the given - ; digest, but only if the router has the given nickname. - ; $Digest=Name -- The router whose identity key hashes to the given - ; digest, but only if the router is Named and has the given - ; nickname. - ; Name -- The Named router with the given nickname, or, if no such - ; router exists, any router whose nickname matches the one given. - ; This is not a safe way to refer to routers, since Named status - ; could under some circumstances change over time. - ServerSpec = LongName / Nickname - - ; Unique identifiers for streams or circuits. Currently, Tor only - ; uses digits, but this may change - StreamID = 1*16 IDChar - CircuitID = 1*16 IDChar - IDChar = ALPHA / DIGIT - - Address = ip4-address / ip6-address / hostname (XXXX Define these) - - ; A "Data" section is a sequence of octets concluded by the terminating - ; sequence CRLF "." CRLF. The terminating sequence may not appear in the - ; body of the data. Leading periods on lines in the data are escaped with - ; an additional leading period as in RFC 2821 section 4.5.2. - Data = *DataLine "." CRLF - DataLine = CRLF / "." 1*LineItem CRLF / NonDotItem *LineItem CRLF - LineItem = NonCR / 1*CR NonCRLF - NonDotItem = NonDotCR / 1*CR NonCRLF - -3. Commands - - All commands are case-insensitive, but most keywords are case-sensitive. - -3.1. SETCONF - - Change the value of one or more configuration variables. The syntax is: - - "SETCONF" 1*(SP keyword ["=" value]) CRLF - value = String / QuotedString - - Tor behaves as though it had just read each of the key-value pairs - from its configuration file. Keywords with no corresponding values have - their configuration values reset to 0 or NULL (use RESETCONF if you want - to set it back to its default). SETCONF is all-or-nothing: if there - is an error in any of the configuration settings, Tor sets none of them. - - Tor responds with a "250 configuration values set" reply on success. - If some of the listed keywords can't be found, Tor replies with a - "552 Unrecognized option" message. Otherwise, Tor responds with a - "513 syntax error in configuration values" reply on syntax error, or a - "553 impossible configuration setting" reply on a semantic error. - - When a configuration option takes multiple values, or when multiple - configuration keys form a context-sensitive group (see GETCONF below), then - setting _any_ of the options in a SETCONF command is taken to reset all of - the others. For example, if two ORBindAddress values are configured, and a - SETCONF command arrives containing a single ORBindAddress value, the new - command's value replaces the two old values. - - Sometimes it is not possible to change configuration options solely by - issuing a series of SETCONF commands, because the value of one of the - configuration options depends on the value of another which has not yet - been set. Such situations can be overcome by setting multiple configuration - options with a single SETCONF command (e.g. SETCONF ORPort=443 - ORListenAddress=9001). - -3.2. RESETCONF - - Remove all settings for a given configuration option entirely, assign - its default value (if any), and then assign the String provided. - Typically the String is left empty, to simply set an option back to - its default. The syntax is: - - "RESETCONF" 1*(SP keyword ["=" String]) CRLF - - Otherwise it behaves like SETCONF above. - -3.3. GETCONF - - Request the value of a configuration variable. The syntax is: - - "GETCONF" 1*(SP keyword) CRLF - - If all of the listed keywords exist in the Tor configuration, Tor replies - with a series of reply lines of the form: - 250 keyword=value - If any option is set to a 'default' value semantically different from an - empty string, Tor may reply with a reply line of the form: - 250 keyword - - Value may be a raw value or a quoted string. Tor will try to use - unquoted values except when the value could be misinterpreted through - not being quoted. - - If some of the listed keywords can't be found, Tor replies with a - "552 unknown configuration keyword" message. - - If an option appears multiple times in the configuration, all of its - key-value pairs are returned in order. - - Some options are context-sensitive, and depend on other options with - different keywords. These cannot be fetched directly. Currently there - is only one such option: clients should use the "HiddenServiceOptions" - virtual keyword to get all HiddenServiceDir, HiddenServicePort, - HiddenServiceNodes, and HiddenServiceExcludeNodes option settings. - -3.4. SETEVENTS - - Request the server to inform the client about interesting events. The - syntax is: - - "SETEVENTS" [SP "EXTENDED"] *(SP EventCode) CRLF - - EventCode = "CIRC" / "STREAM" / "ORCONN" / "BW" / "DEBUG" / - "INFO" / "NOTICE" / "WARN" / "ERR" / "NEWDESC" / "ADDRMAP" / - "AUTHDIR_NEWDESCS" / "DESCCHANGED" / "STATUS_GENERAL" / - "STATUS_CLIENT" / "STATUS_SERVER" / "GUARD" / "NS" / "STREAM_BW" / - "CLIENTS_SEEN" - - Any events *not* listed in the SETEVENTS line are turned off; thus, sending - SETEVENTS with an empty body turns off all event reporting. - - The server responds with a "250 OK" reply on success, and a "552 - Unrecognized event" reply if one of the event codes isn't recognized. (On - error, the list of active event codes isn't changed.) - - If the flag string "EXTENDED" is provided, Tor may provide extra - information with events for this connection; see 4.1 for more information. - NOTE: All events on a given connection will be provided in extended format, - or none. - NOTE: "EXTENDED" is only supported in Tor 0.1.1.9-alpha or later. - - Each event is described in more detail in Section 4.1. - -3.5. AUTHENTICATE - - Sent from the client to the server. The syntax is: - "AUTHENTICATE" [ SP 1*HEXDIG / QuotedString ] CRLF - - The server responds with "250 OK" on success or "515 Bad authentication" if - the authentication cookie is incorrect. Tor closes the connection on an - authentication failure. - - The format of the 'cookie' is implementation-dependent; see 5.1 below for - information on how the standard Tor implementation handles it. - - Before the client has authenticated, no command other than PROTOCOLINFO, - AUTHENTICATE, or QUIT is valid. If the controller sends any other command, - or sends a malformed command, or sends an unsuccessful AUTHENTICATE - command, or sends PROTOCOLINFO more than once, Tor sends an error reply and - closes the connection. - - To prevent some cross-protocol attacks, the AUTHENTICATE command is still - required even if all authentication methods in Tor are disabled. In this - case, the controller should just send "AUTHENTICATE" CRLF. - - (Versions of Tor before 0.1.2.16 and 0.2.0.4-alpha did not close the - connection after an authentication failure.) - -3.6. SAVECONF - - Sent from the client to the server. The syntax is: - "SAVECONF" CRLF - - Instructs the server to write out its config options into its torrc. Server - returns "250 OK" if successful, or "551 Unable to write configuration - to disk" if it can't write the file or some other error occurs. - -3.7. SIGNAL - - Sent from the client to the server. The syntax is: - - "SIGNAL" SP Signal CRLF - - Signal = "RELOAD" / "SHUTDOWN" / "DUMP" / "DEBUG" / "HALT" / - "HUP" / "INT" / "USR1" / "USR2" / "TERM" / "NEWNYM" / - "CLEARDNSCACHE" - - The meaning of the signals are: - - RELOAD -- Reload: reload config items, refetch directory. (like HUP) - SHUTDOWN -- Controlled shutdown: if server is an OP, exit immediately. - If it's an OR, close listeners and exit after 30 seconds. - (like INT) - DUMP -- Dump stats: log information about open connections and - circuits. (like USR1) - DEBUG -- Debug: switch all open logs to loglevel debug. (like USR2) - HALT -- Immediate shutdown: clean up and exit now. (like TERM) - CLEARDNSCACHE -- Forget the client-side cached IPs for all hostnames. - NEWNYM -- Switch to clean circuits, so new application requests - don't share any circuits with old ones. Also clears - the client-side DNS cache. (Tor MAY rate-limit its - response to this signal.) - - The server responds with "250 OK" if the signal is recognized (or simply - closes the socket if it was asked to close immediately), or "552 - Unrecognized signal" if the signal is unrecognized. - -3.8. MAPADDRESS - - Sent from the client to the server. The syntax is: - - "MAPADDRESS" 1*(Address "=" Address SP) CRLF - - The first address in each pair is an "original" address; the second is a - "replacement" address. The client sends this message to the server in - order to tell it that future SOCKS requests for connections to the original - address should be replaced with connections to the specified replacement - address. If the addresses are well-formed, and the server is able to - fulfill the request, the server replies with a 250 message: - 250-OldAddress1=NewAddress1 - 250 OldAddress2=NewAddress2 - - containing the source and destination addresses. If request is - malformed, the server replies with "512 syntax error in command - argument". If the server can't fulfill the request, it replies with - "451 resource exhausted". - - The client may decline to provide a body for the original address, and - instead send a special null address ("0.0.0.0" for IPv4, "::0" for IPv6, or - "." for hostname), signifying that the server should choose the original - address itself, and return that address in the reply. The server - should ensure that it returns an element of address space that is unlikely - to be in actual use. If there is already an address mapped to the - destination address, the server may reuse that mapping. - - If the original address is already mapped to a different address, the old - mapping is removed. If the original address and the destination address - are the same, the server removes any mapping in place for the original - address. - - Example: - C: MAPADDRESS 0.0.0.0=torproject.org 1.2.3.4=tor.freehaven.net - S: 250-127.192.10.10=torproject.org - S: 250 1.2.3.4=tor.freehaven.net - - {Note: This feature is designed to be used to help Tor-ify applications - that need to use SOCKS4 or hostname-less SOCKS5. There are three - approaches to doing this: - 1. Somehow make them use SOCKS4a or SOCKS5-with-hostnames instead. - 2. Use tor-resolve (or another interface to Tor's resolve-over-SOCKS - feature) to resolve the hostname remotely. This doesn't work - with special addresses like x.onion or x.y.exit. - 3. Use MAPADDRESS to map an IP address to the desired hostname, and then - arrange to fool the application into thinking that the hostname - has resolved to that IP. - This functionality is designed to help implement the 3rd approach.} - - Mappings set by the controller last until the Tor process exits: - they never expire. If the controller wants the mapping to last only - a certain time, then it must explicitly un-map the address when that - time has elapsed. - -3.9. GETINFO - - Sent from the client to the server. The syntax is as for GETCONF: - "GETINFO" 1*(SP keyword) CRLF - one or more NL-terminated strings. The server replies with an INFOVALUE - message, or a 551 or 552 error. - - Unlike GETCONF, this message is used for data that are not stored in the Tor - configuration file, and that may be longer than a single line. On success, - one ReplyLine is sent for each requested value, followed by a final 250 OK - ReplyLine. If a value fits on a single line, the format is: - 250-keyword=value - If a value must be split over multiple lines, the format is: - 250+keyword= - value - . - Recognized keys and their values include: - - "version" -- The version of the server's software, including the name - of the software. (example: "Tor 0.0.9.4") - - "config-file" -- The location of Tor's configuration file ("torrc"). - - ["exit-policy/prepend" -- The default exit policy lines that Tor will - *prepend* to the ExitPolicy config option. - -- Never implemented. Useful?] - - "exit-policy/default" -- The default exit policy lines that Tor will - *append* to the ExitPolicy config option. - - "desc/id/" or "desc/name/" -- the latest - server descriptor for a given OR, NUL-terminated. - - "desc-annotations/id/" -- outputs the annotations string - (source, timestamp of arrival, purpose, etc) for the corresponding - descriptor. [First implemented in 0.2.0.13-alpha.] - - "extra-info/digest/" -- the extrainfo document whose digest (in - hex) is . Only available if we're downloading extra-info - documents. - - "ns/id/" or "ns/name/" -- the latest router - status info (v2 directory style) for a given OR. Router status - info is as given in - dir-spec.txt, and reflects the current beliefs of this Tor about the - router in question. Like directory clients, controllers MUST - tolerate unrecognized flags and lines. The published date and - descriptor digest are those believed to be best by this Tor, - not necessarily those for a descriptor that Tor currently has. - [First implemented in 0.1.2.3-alpha.] - - "ns/all" -- Router status info (v2 directory style) for all ORs we - have an opinion about, joined by newlines. [First implemented - in 0.1.2.3-alpha.] - - "ns/purpose/" -- Router status info (v2 directory style) - for all ORs of this purpose. Mostly designed for /ns/purpose/bridge - queries. [First implemented in 0.2.0.13-alpha.] - - "desc/all-recent" -- the latest server descriptor for every router that - Tor knows about. - - "network-status" -- a space-separated list (v1 directory style) - of all known OR identities. This is in the same format as the - router-status line in v1 directories; see dir-spec-v1.txt section - 3 for details. (If VERBOSE_NAMES is enabled, the output will - not conform to dir-spec-v1.txt; instead, the result will be a - space-separated list of LongName, each preceded by a "!" if it is - believed to be not running.) This option is deprecated; use - "ns/all" instead. - - "address-mappings/all" - "address-mappings/config" - "address-mappings/cache" - "address-mappings/control" -- a \r\n-separated list of address - mappings, each in the form of "from-address to-address expiry". - The 'config' key returns those address mappings set in the - configuration; the 'cache' key returns the mappings in the - client-side DNS cache; the 'control' key returns the mappings set - via the control interface; the 'all' target returns the mappings - set through any mechanism. - Expiry is formatted as with ADDRMAP events, except that "expiry" is - always a time in GMT or the string "NEVER"; see section 4.1.7. - First introduced in 0.2.0.3-alpha. - - "addr-mappings/*" -- as for address-mappings/*, but without the - expiry portion of the value. Use of this value is deprecated - since 0.2.0.3-alpha; use address-mappings instead. - - "address" -- the best guess at our external IP address. If we - have no guess, return a 551 error. (Added in 0.1.2.2-alpha) - - "fingerprint" -- the contents of the fingerprint file that Tor - writes as a server, or a 551 if we're not a server currently. - (Added in 0.1.2.3-alpha) - - "circuit-status" - A series of lines as for a circuit status event. Each line is of - the form: - CircuitID SP CircStatus [SP Path] CRLF - - "stream-status" - A series of lines as for a stream status event. Each is of the form: - StreamID SP StreamStatus SP CircID SP Target CRLF - - "orconn-status" - A series of lines as for an OR connection status event. Each is of the - form: - ServerID SP ORStatus CRLF - - "entry-guards" - A series of lines listing the currently chosen entry guards, if any. - Each is of the form: - ServerID2 SP Status [SP ISOTime] CRLF - - Status-with-time = ("unlisted") SP ISOTime - Status = ("up" / "never-connected" / "down" / - "unusable" / "unlisted" ) - - ServerID2 = Nickname / 40*HEXDIG - - [From 0.1.1.4-alpha to 0.1.1.10-alpha, this was called "helper-nodes". - Tor still supports calling it that for now, but support will be - removed in 0.1.3.x.] - - [Older versions of Tor (before 0.1.2.x-final) generated 'down' instead - of unlisted/unusable. Current Tors never generate 'down'.] - - [XXXX ServerID2 differs from ServerID in not prefixing fingerprints - with a $. This is an implementation error. It would be nice to add - the $ back in if we can do so without breaking compatibility.] - - "accounting/enabled" - "accounting/hibernating" - "accounting/bytes" - "accounting/bytes-left" - "accounting/interval-start" - "accounting/interval-wake" - "accounting/interval-end" - Information about accounting status. If accounting is enabled, - "enabled" is 1; otherwise it is 0. The "hibernating" field is "hard" - if we are accepting no data; "soft" if we're accepting no new - connections, and "awake" if we're not hibernating at all. The "bytes" - and "bytes-left" fields contain (read-bytes SP write-bytes), for the - start and the rest of the interval respectively. The 'interval-start' - and 'interval-end' fields are the borders of the current interval; the - 'interval-wake' field is the time within the current interval (if any) - where we plan[ned] to start being active. - - "config/names" - A series of lines listing the available configuration options. Each is - of the form: - OptionName SP OptionType [ SP Documentation ] CRLF - OptionName = Keyword - OptionType = "Integer" / "TimeInterval" / "DataSize" / "Float" / - "Boolean" / "Time" / "CommaList" / "Dependant" / "Virtual" / - "String" / "LineList" - Documentation = Text - - "info/names" - A series of lines listing the available GETINFO options. Each is of - one of these forms: - OptionName SP Documentation CRLF - OptionPrefix SP Documentation CRLF - OptionPrefix = OptionName "/*" - - "events/names" - A space-separated list of all the events supported by this version of - Tor's SETEVENTS. - - "features/names" - A space-separated list of all the events supported by this version of - Tor's USEFEATURE. - - "ip-to-country/*" - Maps IP addresses to 2-letter country codes. For example, - "GETINFO ip-to-country/18.0.0.1" should give "US". - - "next-circuit/IP:port" - XXX todo. - - "dir/status-vote/current/consensus" [added in Tor 0.2.1.6-alpha] - "dir/status/authority" - "dir/status/fp/" - "dir/status/fp/++" - "dir/status/all" - "dir/server/fp/" - "dir/server/fp/++" - "dir/server/d/" - "dir/server/d/++" - "dir/server/authority" - "dir/server/all" - A series of lines listing directory contents, provided according to the - specification for the URLs listed in Section 4.4 of dir-spec.txt. Note - that Tor MUST NOT provide private information, such as descriptors for - routers not marked as general-purpose. When asked for 'authority' - information for which this Tor is not authoritative, Tor replies with - an empty string. - - "status/circuit-established" - "status/enough-dir-info" - "status/good-server-descriptor" - "status/accepted-server-descriptor" - "status/..." - These provide the current internal Tor values for various Tor - states. See Section 4.1.10 for explanations. (Only a few of the - status events are available as getinfo's currently. Let us know if - you want more exposed.) - "status/reachability/or" - 0 or 1, depending on whether we've found our ORPort reachable. - "status/reachability/dir" - 0 or 1, depending on whether we've found our DirPort reachable. - "status/reachability" - "OR=" ("0"/"1") SP "DIR=" ("0"/"1") - Combines status/reachability/*; controllers MUST ignore unrecognized - elements in this entry. - "status/bootstrap-phase" - Returns the most recent bootstrap phase status event - sent. Specifically, it returns a string starting with either - "NOTICE BOOTSTRAP ..." or "WARN BOOTSTRAP ...". Controllers should - use this getinfo when they connect or attach to Tor to learn its - current bootstrap state. - "status/version/recommended" - List of currently recommended versions. - "status/version/current" - Status of the current version. One of: new, old, unrecommended, - recommended, new in series, obsolete. - "status/clients-seen" - A summary of which countries we've seen clients from recently, - formatted the same as the CLIENTS_SEEN status event described in - Section 4.1.14. This GETINFO option is currently available only - for bridge relays. - - Examples: - C: GETINFO version desc/name/moria1 - S: 250+desc/name/moria= - S: [Descriptor for moria] - S: . - S: 250-version=Tor 0.1.1.0-alpha-cvs - S: 250 OK - -3.10. EXTENDCIRCUIT - - Sent from the client to the server. The format is: - "EXTENDCIRCUIT" SP CircuitID SP - ServerSpec *("," ServerSpec) - [SP "purpose=" Purpose] CRLF - - This request takes one of two forms: either the CircuitID is zero, in - which case it is a request for the server to build a new circuit according - to the specified path, or the CircuitID is nonzero, in which case it is a - request for the server to extend an existing circuit with that ID according - to the specified path. - - If CircuitID is 0 and "purpose=" is specified, then the circuit's - purpose is set. Two choices are recognized: "general" and - "controller". If not specified, circuits are created as "general". - - If the request is successful, the server sends a reply containing a - message body consisting of the CircuitID of the (maybe newly created) - circuit. The syntax is "250" SP "EXTENDED" SP CircuitID CRLF. - -3.11. SETCIRCUITPURPOSE - - Sent from the client to the server. The format is: - "SETCIRCUITPURPOSE" SP CircuitID SP Purpose CRLF - - This changes the circuit's purpose. See EXTENDCIRCUIT above for details. - -3.12. SETROUTERPURPOSE - - Sent from the client to the server. The format is: - "SETROUTERPURPOSE" SP NicknameOrKey SP Purpose CRLF - - This changes the descriptor's purpose. See +POSTDESCRIPTOR below - for details. - - NOTE: This command was disabled and made obsolete as of Tor - 0.2.0.8-alpha. It doesn't exist anymore, and is listed here only for - historical interest. - -3.13. ATTACHSTREAM - - Sent from the client to the server. The syntax is: - "ATTACHSTREAM" SP StreamID SP CircuitID [SP "HOP=" HopNum] CRLF - - This message informs the server that the specified stream should be - associated with the specified circuit. Each stream may be associated with - at most one circuit, and multiple streams may share the same circuit. - Streams can only be attached to completed circuits (that is, circuits that - have sent a circuit status 'BUILT' event or are listed as built in a - GETINFO circuit-status request). - - If the circuit ID is 0, responsibility for attaching the given stream is - returned to Tor. - - If HOP=HopNum is specified, Tor will choose the HopNumth hop in the - circuit as the exit node, rather than the last node in the circuit. - Hops are 1-indexed; generally, it is not permitted to attach to hop 1. - - Tor responds with "250 OK" if it can attach the stream, 552 if the circuit - or stream didn't exist, or 551 if the stream couldn't be attached for - another reason. - - {Implementation note: Tor will close unattached streams by itself, - roughly two minutes after they are born. Let the developers know if - that turns out to be a problem.} - - {Implementation note: By default, Tor automatically attaches streams to - circuits itself, unless the configuration variable - "__LeaveStreamsUnattached" is set to "1". Attempting to attach streams - via TC when "__LeaveStreamsUnattached" is false may cause a race between - Tor and the controller, as both attempt to attach streams to circuits.} - - {Implementation note: You can try to attachstream to a stream that - has already sent a connect or resolve request but hasn't succeeded - yet, in which case Tor will detach the stream from its current circuit - before proceeding with the new attach request.} - -3.14. POSTDESCRIPTOR - - Sent from the client to the server. The syntax is: - "+POSTDESCRIPTOR" [SP "purpose=" Purpose] [SP "cache=" Cache] - CRLF Descriptor CRLF "." CRLF - - This message informs the server about a new descriptor. If Purpose is - specified, it must be either "general", "controller", or "bridge", - else we return a 552 error. The default is "general". - - If Cache is specified, it must be either "no" or "yes", else we - return a 552 error. If Cache is not specified, Tor will decide for - itself whether it wants to cache the descriptor, and controllers - must not rely on its choice. - - The descriptor, when parsed, must contain a number of well-specified - fields, including fields for its nickname and identity. - - If there is an error in parsing the descriptor, the server must send a - "554 Invalid descriptor" reply. If the descriptor is well-formed but - the server chooses not to add it, it must reply with a 251 message - whose body explains why the server was not added. If the descriptor - is added, Tor replies with "250 OK". - -3.15. REDIRECTSTREAM - - Sent from the client to the server. The syntax is: - "REDIRECTSTREAM" SP StreamID SP Address [SP Port] CRLF - - Tells the server to change the exit address on the specified stream. If - Port is specified, changes the destination port as well. No remapping - is performed on the new provided address. - - To be sure that the modified address will be used, this event must be sent - after a new stream event is received, and before attaching this stream to - a circuit. - - Tor replies with "250 OK" on success. - -3.16. CLOSESTREAM - - Sent from the client to the server. The syntax is: - - "CLOSESTREAM" SP StreamID SP Reason *(SP Flag) CRLF - - Tells the server to close the specified stream. The reason should be one - of the Tor RELAY_END reasons given in tor-spec.txt, as a decimal. Flags is - not used currently; Tor servers SHOULD ignore unrecognized flags. Tor may - hold the stream open for a while to flush any data that is pending. - - Tor replies with "250 OK" on success, or a 512 if there aren't enough - arguments, or a 552 if it doesn't recognize the StreamID or reason. - -3.17. CLOSECIRCUIT - - The syntax is: - CLOSECIRCUIT SP CircuitID *(SP Flag) CRLF - Flag = "IfUnused" - - Tells the server to close the specified circuit. If "IfUnused" is - provided, do not close the circuit unless it is unused. - - Other flags may be defined in the future; Tor SHOULD ignore unrecognized - flags. - - Tor replies with "250 OK" on success, or a 512 if there aren't enough - arguments, or a 552 if it doesn't recognize the CircuitID. - -3.18. QUIT - - Tells the server to hang up on this controller connection. This command - can be used before authenticating. - -3.19. USEFEATURE - - The syntax is: - - "USEFEATURE" *(SP FeatureName) CRLF - FeatureName = 1*(ALPHA / DIGIT / "_" / "-") - - Sometimes extensions to the controller protocol break compatibility with - older controllers. In this case, whenever possible, the extensions are - first included in Tor disabled by default, and only enabled on a given - controller connection when the "USEFEATURE" command is given. Once a - "USEFEATURE" command is given, it applies to all subsequent interactions on - the same connection; to disable an enabled feature, a new controller - connection must be opened. - - This is a forward-compatibility mechanism; each feature will eventually - become a regular part of the control protocol in some future version of Tor. - Tor will ignore a request to use any feature that is already on by default. - Tor will give a "552" error if any requested feature is not recognized. - - Feature names are case-insensitive. - - EXTENDED_EVENTS - - Same as passing 'EXTENDED' to SETEVENTS; this is the preferred way to - request the extended event syntax. - - This will not be always-enabled until at least two stable releases - after 0.1.2.3-alpha, the release where it was first used for - anything. - - VERBOSE_NAMES - - Instead of ServerID as specified above, the controller should - identify ORs by LongName in events and GETINFO results. This format is - strictly more informative: rather than including Nickname for - known Named routers and Fingerprint for unknown or unNamed routers, the - LongName format includes a Fingerprint, an indication of Named status, - and a Nickname (if one is known). - - This will not be always-enabled until at least two stable releases - after 0.1.2.2-alpha, the release where it was first available. - -3.20. RESOLVE - - The syntax is - "RESOLVE" *Option *Address CRLF - Option = "mode=reverse" - Address = a hostname or IPv4 address - - This command launches a remote hostname lookup request for every specified - request (or reverse lookup if "mode=reverse" is specified). Note that the - request is done in the background: to see the answers, your controller will - need to listen for ADDRMAP events; see 4.1.7 below. - - [Added in Tor 0.2.0.3-alpha] - -3.21. PROTOCOLINFO - - The syntax is: - "PROTOCOLINFO" *(SP PIVERSION) CRLF - - The server reply format is: - "250-PROTOCOLINFO" SP PIVERSION CRLF *InfoLine "250 OK" CRLF - - InfoLine = AuthLine / VersionLine / OtherLine - - AuthLine = "250-AUTH" SP "METHODS=" AuthMethod *(",")AuthMethod - *(SP "COOKIEFILE=" AuthCookieFile) CRLF - VersionLine = "250-VERSION" SP "Tor=" TorVersion [SP Arguments] CRLF - - AuthMethod = - "NULL" / ; No authentication is required - "HASHEDPASSWORD" / ; A controller must supply the original password - "COOKIE" / ; A controller must supply the contents of a cookie - - AuthCookieFile = QuotedString - TorVersion = QuotedString - - OtherLine = "250-" Keyword [SP Arguments] CRLF - - PIVERSION: 1*DIGIT - - Tor MAY give its InfoLines in any order; controllers MUST ignore InfoLines - with keywords they do not recognize. Controllers MUST ignore extraneous - data on any InfoLine. - - PIVERSION is there in case we drastically change the syntax one day. For - now it should always be "1". Controllers MAY provide a list of the - protocolinfo versions they support; Tor MAY select a version that the - controller does not support. - - AuthMethod is used to specify one or more control authentication - methods that Tor currently accepts. - - AuthCookieFile specifies the absolute path and filename of the - authentication cookie that Tor is expecting and is provided iff - the METHODS field contains the method "COOKIE". Controllers MUST handle - escape sequences inside this string. - - The VERSION line contains the Tor version. - - [Unlike other commands besides AUTHENTICATE, PROTOCOLINFO may be used (but - only once!) before AUTHENTICATE.] - - [PROTOCOLINFO was not supported before Tor 0.2.0.5-alpha.] - -4. Replies - - Reply codes follow the same 3-character format as used by SMTP, with the - first character defining a status, the second character defining a - subsystem, and the third designating fine-grained information. - - The TC protocol currently uses the following first characters: - - 2yz Positive Completion Reply - The command was successful; a new request can be started. - - 4yz Temporary Negative Completion reply - The command was unsuccessful but might be reattempted later. - - 5yz Permanent Negative Completion Reply - The command was unsuccessful; the client should not try exactly - that sequence of commands again. - - 6yz Asynchronous Reply - Sent out-of-order in response to an earlier SETEVENTS command. - - The following second characters are used: - - x0z Syntax - Sent in response to ill-formed or nonsensical commands. - - x1z Protocol - Refers to operations of the Tor Control protocol. - - x5z Tor - Refers to actual operations of Tor system. - - The following codes are defined: - - 250 OK - 251 Operation was unnecessary - [Tor has declined to perform the operation, but no harm was done.] - - 451 Resource exhausted - - 500 Syntax error: protocol - - 510 Unrecognized command - 511 Unimplemented command - 512 Syntax error in command argument - 513 Unrecognized command argument - 514 Authentication required - 515 Bad authentication - - 550 Unspecified Tor error - - 551 Internal error - [Something went wrong inside Tor, so that the client's - request couldn't be fulfilled.] - - 552 Unrecognized entity - [A configuration key, a stream ID, circuit ID, event, - mentioned in the command did not actually exist.] - - 553 Invalid configuration value - [The client tried to set a configuration option to an - incorrect, ill-formed, or impossible value.] - - 554 Invalid descriptor - - 555 Unmanaged entity - - 650 Asynchronous event notification - - Unless specified to have specific contents, the human-readable messages - in error replies should not be relied upon to match those in this document. - -4.1. Asynchronous events - - These replies can be sent after a corresponding SETEVENTS command has been - received. They will not be interleaved with other Reply elements, but they - can appear between a command and its corresponding reply. For example, - this sequence is possible: - - C: SETEVENTS CIRC - S: 250 OK - C: GETCONF SOCKSPORT ORPORT - S: 650 CIRC 1000 EXTENDED moria1,moria2 - S: 250-SOCKSPORT=9050 - S: 250 ORPORT=0 - - But this sequence is disallowed: - C: SETEVENTS CIRC - S: 250 OK - C: GETCONF SOCKSPORT ORPORT - S: 250-SOCKSPORT=9050 - S: 650 CIRC 1000 EXTENDED moria1,moria2 - S: 250 ORPORT=0 - - Clients MUST tolerate more arguments in an asynchonous reply than - expected, and MUST tolerate more lines in an asynchronous reply than - expected. For instance, a client that expects a CIRC message like: - 650 CIRC 1000 EXTENDED moria1,moria2 - must tolerate: - 650-CIRC 1000 EXTENDED moria1,moria2 0xBEEF - 650-EXTRAMAGIC=99 - 650 ANONYMITY=high - - If clients ask for extended events, then each event line as specified below - will be followed by additional extensions. Additional lines will be of the - form - "650" ("-"/" ") KEYWORD ["=" ARGUMENTS] CRLF - Additional arguments will be of the form - SP KEYWORD ["=" ( QuotedString / * NonSpDquote ) ] - Such clients MUST tolerate lines with keywords they do not recognize. - -4.1.1. Circuit status changed - - The syntax is: - - "650" SP "CIRC" SP CircuitID SP CircStatus [SP Path] - [SP "REASON=" Reason [SP "REMOTE_REASON=" Reason]] CRLF - - CircStatus = - "LAUNCHED" / ; circuit ID assigned to new circuit - "BUILT" / ; all hops finished, can now accept streams - "EXTENDED" / ; one more hop has been completed - "FAILED" / ; circuit closed (was not built) - "CLOSED" ; circuit closed (was built) - - Path = ServerID *("," ServerID) - - Reason = "NONE" / "TORPROTOCOL" / "INTERNAL" / "REQUESTED" / - "HIBERNATING" / "RESOURCELIMIT" / "CONNECTFAILED" / - "OR_IDENTITY" / "OR_CONN_CLOSED" / "TIMEOUT" / - "FINISHED" / "DESTROYED" / "NOPATH" / "NOSUCHSERVICE" - - The path is provided only when the circuit has been extended at least one - hop. - - The "REASON" field is provided only for FAILED and CLOSED events, and only - if extended events are enabled (see 3.19). Clients MUST accept reasons - not listed above. Reasons are as given in tor-spec.txt, except for: - - NOPATH (Not enough nodes to make circuit) - - The "REMOTE_REASON" field is provided only when we receive a DESTROY or - TRUNCATE cell, and only if extended events are enabled. It contains the - actual reason given by the remote OR for closing the circuit. Clients MUST - accept reasons not listed above. Reasons are as listed in tor-spec.txt. - -4.1.2. Stream status changed - - The syntax is: - - "650" SP "STREAM" SP StreamID SP StreamStatus SP CircID SP Target - [SP "REASON=" Reason [ SP "REMOTE_REASON=" Reason ]] - [SP "SOURCE=" Source] [ SP "SOURCE_ADDR=" Address ":" Port ] - [SP "PURPOSE=" Purpose] - CRLF - - StreamStatus = - "NEW" / ; New request to connect - "NEWRESOLVE" / ; New request to resolve an address - "REMAP" / ; Address re-mapped to another - "SENTCONNECT" / ; Sent a connect cell along a circuit - "SENTRESOLVE" / ; Sent a resolve cell along a circuit - "SUCCEEDED" / ; Received a reply; stream established - "FAILED" / ; Stream failed and not retriable - "CLOSED" / ; Stream closed - "DETACHED" ; Detached from circuit; still retriable - - Target = Address ":" Port - - The circuit ID designates which circuit this stream is attached to. If - the stream is unattached, the circuit ID "0" is given. - - Reason = "MISC" / "RESOLVEFAILED" / "CONNECTREFUSED" / - "EXITPOLICY" / "DESTROY" / "DONE" / "TIMEOUT" / - "HIBERNATING" / "INTERNAL"/ "RESOURCELIMIT" / - "CONNRESET" / "TORPROTOCOL" / "NOTDIRECTORY" / "END" - - The "REASON" field is provided only for FAILED, CLOSED, and DETACHED - events, and only if extended events are enabled (see 3.19). Clients MUST - accept reasons not listed above. Reasons are as given in tor-spec.txt, - except for: - - END (We received a RELAY_END cell from the other side of this - stream.) - [XXXX document more. -NM] - - The "REMOTE_REASON" field is provided only when we receive a RELAY_END - cell, and only if extended events are enabled. It contains the actual - reason given by the remote OR for closing the stream. Clients MUST accept - reasons not listed above. Reasons are as listed in tor-spec.txt. - - "REMAP" events include a Source if extended events are enabled: - Source = "CACHE" / "EXIT" - Clients MUST accept sources not listed above. "CACHE" is given if - the Tor client decided to remap the address because of a cached - answer, and "EXIT" is given if the remote node we queried gave us - the new address as a response. - - The "SOURCE_ADDR" field is included with NEW and NEWRESOLVE events if - extended events are enabled. It indicates the address and port - that requested the connection, and can be (e.g.) used to look up the - requesting program. - - Purpose = "DIR_FETCH" / "UPLOAD_DESC" / "DNS_REQUEST" / - "USER" / "DIRPORT_TEST" - - The "PURPOSE" field is provided only for NEW and NEWRESOLVE events, and - only if extended events are enabled (see 3.19). Clients MUST accept - purposes not listed above. - -4.1.3. OR Connection status changed - - The syntax is: - "650" SP "ORCONN" SP (ServerID / Target) SP ORStatus [ SP "REASON=" - Reason ] [ SP "NCIRCS=" NumCircuits ] CRLF - - ORStatus = "NEW" / "LAUNCHED" / "CONNECTED" / "FAILED" / "CLOSED" - - NEW is for incoming connections, and LAUNCHED is for outgoing - connections. CONNECTED means the TLS handshake has finished (in - either direction). FAILED means a connection is being closed that - hasn't finished its handshake, and CLOSED is for connections that - have handshaked. - - A ServerID is specified unless it's a NEW connection, in which - case we don't know what server it is yet, so we use Address:Port. - - If extended events are enabled (see 3.19), optional reason and - circuit counting information is provided for CLOSED and FAILED - events. - - Reason = "MISC" / "DONE" / "CONNECTREFUSED" / - "IDENTITY" / "CONNECTRESET" / "TIMEOUT" / "NOROUTE" / - "IOERROR" / "RESOURCELIMIT" - - NumCircuits counts both established and pending circuits. - -4.1.4. Bandwidth used in the last second - - The syntax is: - "650" SP "BW" SP BytesRead SP BytesWritten *(SP Type "=" Num) CRLF - BytesRead = 1*DIGIT - BytesWritten = 1*DIGIT - Type = "DIR" / "OR" / "EXIT" / "APP" / ... - Num = 1*DIGIT - - BytesRead and BytesWritten are the totals. [In a future Tor version, - we may also include a breakdown of the connection types that used - bandwidth this second (not implemented yet).] - -4.1.5. Log messages - - The syntax is: - "650" SP Severity SP ReplyText CRLF - or - "650+" Severity CRLF Data 650 SP "OK" CRLF - - Severity = "DEBUG" / "INFO" / "NOTICE" / "WARN"/ "ERR" - -4.1.6. New descriptors available - - Syntax: - "650" SP "NEWDESC" 1*(SP ServerID) CRLF - -4.1.7. New Address mapping - - Syntax: - "650" SP "ADDRMAP" SP Address SP NewAddress SP Expiry - [SP Error] SP GMTExpiry CRLF - - NewAddress = Address / "" - Expiry = DQUOTE ISOTime DQUOTE / "NEVER" - - Error = "error=" ErrorCode - ErrorCode = XXXX - GMTExpiry = "EXPIRES=" DQUOTE IsoTime DQUOTE - - Error and GMTExpiry are only provided if extended events are enabled. - - Expiry is expressed as the local time (rather than GMT). This is a bug, - left in for backward compatibility; new code should look at GMTExpiry - instead. - - These events are generated when a new address mapping is entered in the - cache, or when the answer for a RESOLVE command is found. - -4.1.8. Descriptors uploaded to us in our role as authoritative dirserver - - Syntax: - "650" "+" "AUTHDIR_NEWDESCS" CRLF Action CRLF Message CRLF - Descriptor CRLF "." CRLF "650" SP "OK" CRLF - Action = "ACCEPTED" / "DROPPED" / "REJECTED" - Message = Text - -4.1.9. Our descriptor changed - - Syntax: - "650" SP "DESCCHANGED" CRLF - - [First added in 0.1.2.2-alpha.] - -4.1.10. Status events - - Status events (STATUS_GENERAL, STATUS_CLIENT, and STATUS_SERVER) are sent - based on occurrences in the Tor process pertaining to the general state of - the program. Generally, they correspond to log messages of severity Notice - or higher. They differ from log messages in that their format is a - specified interface. - - Syntax: - "650" SP StatusType SP StatusSeverity SP StatusAction - [SP StatusArguments] CRLF - - StatusType = "STATUS_GENERAL" / "STATUS_CLIENT" / "STATUS_SERVER" - StatusSeverity = "NOTICE" / "WARN" / "ERR" - StatusAction = 1*ALPHA - StatusArguments = StatusArgument *(SP StatusArgument) - StatusArgument = StatusKeyword '=' StatusValue - StatusKeyword = 1*(ALNUM / "_") - StatusValue = 1*(ALNUM / '_') / QuotedString - - Action is a string, and Arguments is a series of keyword=value - pairs on the same line. Values may be space-terminated strings, - or quoted strings. - - These events are always produced with EXTENDED_EVENTS and - VERBOSE_NAMES; see the explanations in the USEFEATURE section - for details. - - Controllers MUST tolerate unrecognized actions, MUST tolerate - unrecognized arguments, MUST tolerate missing arguments, and MUST - tolerate arguments that arrive in any order. - - Each event description below is accompanied by a recommendation for - controllers. These recommendations are suggestions only; no controller - is required to implement them. - - Compatibility note: versions of Tor before 0.2.0.22-rc incorrectly - generated "STATUS_SERVER" as "STATUS_SEVER". To be compatible with those - versions, tools should accept both. - - Actions for STATUS_GENERAL events can be as follows: - - CLOCK_JUMPED - "TIME=NUM" - Tor spent enough time without CPU cycles that it has closed all - its circuits and will establish them anew. This typically - happens when a laptop goes to sleep and then wakes up again. It - also happens when the system is swapping so heavily that Tor is - starving. The "time" argument specifies the number of seconds Tor - thinks it was unconscious for (or alternatively, the number of - seconds it went back in time). - - This status event is sent as NOTICE severity normally, but WARN - severity if Tor is acting as a server currently. - - {Recommendation for controller: ignore it, since we don't really - know what the user should do anyway. Hm.} - - DANGEROUS_VERSION - "CURRENT=version" - "REASON=NEW/OBSOLETE/UNRECOMMENDED" - "RECOMMENDED=\"version, version, ...\"" - Tor has found that directory servers don't recommend its version of - the Tor software. RECOMMENDED is a comma-and-space-separated string - of Tor versions that are recommended. REASON is NEW if this version - of Tor is newer than any recommended version, OBSOLETE if - this version of Tor is older than any recommended version, and - UNRECOMMENDED if some recommended versions of Tor are newer and - some are older than this version. (The "OBSOLETE" reason was called - "OLD" from Tor 0.1.2.3-alpha up to and including 0.2.0.12-alpha.) - - {Controllers may want to suggest that the user upgrade OLD or - UNRECOMMENDED versions. NEW versions may be known-insecure, or may - simply be development versions.} - - TOO_MANY_CONNECTIONS - "CURRENT=NUM" - Tor has reached its ulimit -n or whatever the native limit is on file - descriptors or sockets. CURRENT is the number of sockets Tor - currently has open. The user should really do something about - this. The "current" argument shows the number of connections currently - open. - - {Controllers may recommend that the user increase the limit, or - increase it for them. Recommendations should be phrased in an - OS-appropriate way and automated when possible.} - - BUG - "REASON=STRING" - Tor has encountered a situation that its developers never expected, - and the developers would like to learn that it happened. Perhaps - the controller can explain this to the user and encourage her to - file a bug report? - - {Controllers should log bugs, but shouldn't annoy the user in case a - bug appears frequently.} - - CLOCK_SKEW - SKEW="+" / "-" SECONDS - MIN_SKEW="+" / "-" SECONDS. - SOURCE="DIRSERV:" IP ":" Port / - "NETWORKSTATUS:" IP ":" Port / - "OR:" IP ":" Port / - "CONSENSUS" - If "SKEW" is present, it's an estimate of how far we are from the - time declared in the source. (In other words, if we're an hour in - the past, the value is -3600.) "MIN_SKEW" is present, it's a lower - bound. If the source is a DIRSERV, we got the current time from a - connection to a dirserver. If the source is a NETWORKSTATUS, we - decided we're skewed because we got a v2 networkstatus from far in - the future. If the source is OR, the skew comes from a NETINFO - cell from a connection to another relay. If the source is - CONSENSUS, we decided we're skewed because we got a networkstatus - consensus from the future. - - {Tor should send this message to controllers when it thinks the - skew is so high that it will interfere with proper Tor operation. - Controllers shouldn't blindly adjust the clock, since the more - accurate source of skew info (DIRSERV) is currently - unauthenticated.} - - BAD_LIBEVENT - "METHOD=" libevent method - "VERSION=" libevent version - "BADNESS=" "BROKEN" / "BUGGY" / "SLOW" - "RECOVERED=" "NO" / "YES" - Tor knows about bugs in using the configured event method in this - version of libevent. "BROKEN" libevents won't work at all; - "BUGGY" libevents might work okay; "SLOW" libevents will work - fine, but not quickly. If "RECOVERED" is YES, Tor managed to - switch to a more reliable (but probably slower!) libevent method. - - {Controllers may want to warn the user if this event occurs, though - generally it's the fault of whoever built the Tor binary and there's - not much the user can do besides upgrade libevent or upgrade the - binary.} - - DIR_ALL_UNREACHABLE - Tor believes that none of the known directory servers are - reachable -- this is most likely because the local network is - down or otherwise not working, and might help to explain for the - user why Tor appears to be broken. - - {Controllers may want to warn the user if this event occurs; further - action is generally not possible.} - - CONSENSUS_ARRIVED - Tor has received and validated a new consensus networkstatus. - (This event can be delayed a little while after the consensus - is received, if Tor needs to fetch certificates.) - - Actions for STATUS_CLIENT events can be as follows: - - BOOTSTRAP - "PROGRESS=" num - "TAG=" Keyword - "SUMMARY=" String - ["WARNING=" String - "REASON=" Keyword - "COUNT=" num - "RECOMMENDATION=" Keyword - ] - - Tor has made some progress at establishing a connection to the - Tor network, fetching directory information, or making its first - circuit; or it has encountered a problem while bootstrapping. This - status event is especially useful for users with slow connections - or with connectivity problems. - - "Progress" gives a number between 0 and 100 for how far through - the bootstrapping process we are. "Summary" is a string that can - be displayed to the user to describe the *next* task that Tor - will tackle, i.e., the task it is working on after sending the - status event. "Tag" is a string that controllers can use to - recognize bootstrap phases, if they want to do something smarter - than just blindly displaying the summary string; see Section 5 - for the current tags that Tor issues. - - The StatusSeverity describes whether this is a normal bootstrap - phase (severity notice) or an indication of a bootstrapping - problem (severity warn). - - For bootstrap problems, we include the same progress, tag, and - summary values as we would for a normal bootstrap event, but we - also include "warning", "reason", "count", and "recommendation" - key/value combos. The "count" number tells how many bootstrap - problems there have been so far at this phase. The "reason" - string lists one of the reasons allowed in the ORCONN event. The - "warning" argument string with any hints Tor has to offer about - why it's having troubles bootstrapping. - - The "reason" values are long-term-stable controller-facing tags to - identify particular issues in a bootstrapping step. The warning - strings, on the other hand, are human-readable. Controllers - SHOULD NOT rely on the format of any warning string. Currently - the possible values for "recommendation" are either "ignore" or - "warn" -- if ignore, the controller can accumulate the string in - a pile of problems to show the user if the user asks; if warn, - the controller should alert the user that Tor is pretty sure - there's a bootstrapping problem. - - Currently Tor uses recommendation=ignore for the first - nine bootstrap problem reports for a given phase, and then - uses recommendation=warn for subsequent problems at that - phase. Hopefully this is a good balance between tolerating - occasional errors and reporting serious problems quickly. - - ENOUGH_DIR_INFO - Tor now knows enough network-status documents and enough server - descriptors that it's going to start trying to build circuits now. - - {Controllers may want to use this event to decide when to indicate - progress to their users, but should not interrupt the user's browsing - to tell them so.} - - NOT_ENOUGH_DIR_INFO - We discarded expired statuses and router descriptors to fall - below the desired threshold of directory information. We won't - try to build any circuits until ENOUGH_DIR_INFO occurs again. - - {Controllers may want to use this event to decide when to indicate - progress to their users, but should not interrupt the user's browsing - to tell them so.} - - CIRCUIT_ESTABLISHED - Tor is able to establish circuits for client use. This event will - only be sent if we just built a circuit that changed our mind -- - that is, prior to this event we didn't know whether we could - establish circuits. - - {Suggested use: controllers can notify their users that Tor is - ready for use as a client once they see this status event. [Perhaps - controllers should also have a timeout if too much time passes and - this event hasn't arrived, to give tips on how to troubleshoot. - On the other hand, hopefully Tor will send further status events - if it can identify the problem.]} - - CIRCUIT_NOT_ESTABLISHED - "REASON=" "EXTERNAL_ADDRESS" / "DIR_ALL_UNREACHABLE" / "CLOCK_JUMPED" - We are no longer confident that we can build circuits. The "reason" - keyword provides an explanation: which other status event type caused - our lack of confidence. - - {Controllers may want to use this event to decide when to indicate - progress to their users, but should not interrupt the user's browsing - to do so.} - [Note: only REASON=CLOCK_JUMPED is implemented currently.] - - DANGEROUS_PORT - "PORT=" port - "RESULT=" "REJECT" / "WARN" - A stream was initiated to a port that's commonly used for - vulnerable-plaintext protocols. If the Result is "reject", we - refused the connection; whereas if it's "warn", we allowed it. - - {Controllers should warn their users when this occurs, unless they - happen to know that the application using Tor is in fact doing so - correctly (e.g., because it is part of a distributed bundle). They - might also want some sort of interface to let the user configure - their RejectPlaintextPorts and WarnPlaintextPorts config options.} - - DANGEROUS_SOCKS - "PROTOCOL=" "SOCKS4" / "SOCKS5" - "ADDRESS=" IP:port - A connection was made to Tor's SOCKS port using one of the SOCKS - approaches that doesn't support hostnames -- only raw IP addresses. - If the client application got this address from gethostbyname(), - it may be leaking target addresses via DNS. - - {Controllers should warn their users when this occurs, unless they - happen to know that the application using Tor is in fact doing so - correctly (e.g., because it is part of a distributed bundle).} - - SOCKS_UNKNOWN_PROTOCOL - "DATA=string" - A connection was made to Tor's SOCKS port that tried to use it - for something other than the SOCKS protocol. Perhaps the user is - using Tor as an HTTP proxy? The DATA is the first few characters - sent to Tor on the SOCKS port. - - {Controllers may want to warn their users when this occurs: it - indicates a misconfigured application.} - - SOCKS_BAD_HOSTNAME - "HOSTNAME=QuotedString" - Some application gave us a funny-looking hostname. Perhaps - it is broken? In any case it won't work with Tor and the user - should know. - - {Controllers may want to warn their users when this occurs: it - usually indicates a misconfigured application.} - - Actions for STATUS_SERVER can be as follows: - - EXTERNAL_ADDRESS - "ADDRESS=IP" - "HOSTNAME=NAME" - "METHOD=CONFIGURED/DIRSERV/RESOLVED/INTERFACE/GETHOSTNAME" - Our best idea for our externally visible IP has changed to 'IP'. - If 'HOSTNAME' is present, we got the new IP by resolving 'NAME'. If the - method is 'CONFIGURED', the IP was given verbatim as a configuration - option. If the method is 'RESOLVED', we resolved the Address - configuration option to get the IP. If the method is 'GETHOSTNAME', - we resolved our hostname to get the IP. If the method is 'INTERFACE', - we got the address of one of our network interfaces to get the IP. If - the method is 'DIRSERV', a directory server told us a guess for what - our IP might be. - - {Controllers may want to record this info and display it to the user.} - - CHECKING_REACHABILITY - "ORADDRESS=IP:port" - "DIRADDRESS=IP:port" - We're going to start testing the reachability of our external OR port - or directory port. - - {This event could affect the controller's idea of server status, but - the controller should not interrupt the user to tell them so.} - - REACHABILITY_SUCCEEDED - "ORADDRESS=IP:port" - "DIRADDRESS=IP:port" - We successfully verified the reachability of our external OR port or - directory port (depending on which of ORADDRESS or DIRADDRESS is - given.) - - {This event could affect the controller's idea of server status, but - the controller should not interrupt the user to tell them so.} - - GOOD_SERVER_DESCRIPTOR - We successfully uploaded our server descriptor to at least one - of the directory authorities, with no complaints. - - {Originally, the goal of this event was to declare "every authority - has accepted the descriptor, so there will be no complaints - about it." But since some authorities might be offline, it's - harder to get certainty than we had thought. As such, this event - is equivalent to ACCEPTED_SERVER_DESCRIPTOR below. Controllers - should just look at ACCEPTED_SERVER_DESCRIPTOR and should ignore - this event for now.} - - NAMESERVER_STATUS - "NS=addr" - "STATUS=" "UP" / "DOWN" - "ERR=" message - One of our nameservers has changed status. - - {This event could affect the controller's idea of server status, but - the controller should not interrupt the user to tell them so.} - - NAMESERVER_ALL_DOWN - All of our nameservers have gone down. - - {This is a problem; if it happens often without the nameservers - coming up again, the user needs to configure more or better - nameservers.} - - DNS_HIJACKED - Our DNS provider is providing an address when it should be saying - "NOTFOUND"; Tor will treat the address as a synonym for "NOTFOUND". - - {This is an annoyance; controllers may want to tell admins that their - DNS provider is not to be trusted.} - - DNS_USELESS - Our DNS provider is giving a hijacked address instead of well-known - websites; Tor will not try to be an exit node. - - {Controllers could warn the admin if the server is running as an - exit server: the admin needs to configure a good DNS server. - Alternatively, this happens a lot in some restrictive environments - (hotels, universities, coffeeshops) when the user hasn't registered.} - - BAD_SERVER_DESCRIPTOR - "DIRAUTH=addr:port" - "REASON=string" - A directory authority rejected our descriptor. Possible reasons - include malformed descriptors, incorrect keys, highly skewed clocks, - and so on. - - {Controllers should warn the admin, and try to cope if they can.} - - ACCEPTED_SERVER_DESCRIPTOR - "DIRAUTH=addr:port" - A single directory authority accepted our descriptor. - // actually notice - - {This event could affect the controller's idea of server status, but - the controller should not interrupt the user to tell them so.} - - REACHABILITY_FAILED - "ORADDRESS=IP:port" - "DIRADDRESS=IP:port" - We failed to connect to our external OR port or directory port - successfully. - - {This event could affect the controller's idea of server status. The - controller should warn the admin and suggest reasonable steps to take.} - -4.1.11. Our set of guard nodes has changed - - Syntax: - "650" SP "GUARD" SP Type SP Name SP Status ... CRLF - Type = "ENTRY" - Name = The (possibly verbose) nickname of the guard affected. - Status = "NEW" | "UP" | "DOWN" | "BAD" | "GOOD" | "DROPPED" - - [explain states. XXX] - -4.1.12. Network status has changed - - Syntax: - "650" "+" "NS" CRLF 1*NetworkStatus "." CRLF "650" SP "OK" CRLF - - The event is used whenever our local view of a relay status changes. - This happens when we get a new v3 consensus (in which case the entries - we see are a duplicate of what we see in the NEWCONSENSUS event, - below), but it also happens when we decide to mark a relay as up or - down in our local status, for example based on connection attempts. - - [First added in 0.1.2.3-alpha] - -4.1.13. Bandwidth used on an application stream - - The syntax is: - "650" SP "STREAM_BW" SP StreamID SP BytesRead SP BytesWritten CRLF - BytesRead = 1*DIGIT - BytesWritten = 1*DIGIT - - BytesRead and BytesWritten are the number of bytes read and written since - the last STREAM_BW event on this stream. These events are generated about - once per second per stream; no events are generated for streams that have - not read or written. - - These events apply only to streams entering Tor (such as on a SOCKSPort, - TransPort, or so on). They are not generated for exiting streams. - -4.1.14. Per-country client stats - - The syntax is: - "650" SP "CLIENTS_SEEN" SP TimeStarted SP CountrySummary CRLF - - We just generated a new summary of which countries we've seen clients - from recently. The controller could display this for the user, e.g. - in their "relay" configuration window, to give them a sense that they - are actually being useful. - - Currently only bridge relays will receive this event, but once we figure - out how to sufficiently aggregate and sanitize the client counts on - main relays, we might start sending these events in other cases too. - - TimeStarted is a quoted string indicating when the reported summary - counts from (in GMT). - - The CountrySummary keyword has as its argument a comma-separated - set of "countrycode=count" pairs. For example, - 650-CLIENTS_SEEN TimeStarted="Thu Dec 25 23:50:43 EST 2008" - 650 CountrySummary=us=16,de=8,uk=8 -[XXX Matt Edman informs me that the time format above is wrong. -RD] - -4.1.15. New consensus networkstatus has arrived. - - The syntax is: - "650" "+" "NEWCONSENSUS" CRLF 1*NetworkStatus "." CRLF "650" SP - "OK" CRLF - - A new consensus networkstatus has arrived. We include NS-style lines for - every relay in the consensus. NEWCONSENSUS is a separate event from the - NS event, because the list here represents every usable relay: so any - relay *not* mentioned in this list is implicitly no longer recommended. - - [First added in 0.2.1.13-alpha] - -5. Implementation notes - -5.1. Authentication - - If the control port is open and no authentication operation is enabled, Tor - trusts any local user that connects to the control port. This is generally - a poor idea. - - If the 'CookieAuthentication' option is true, Tor writes a "magic cookie" - file named "control_auth_cookie" into its data directory. To authenticate, - the controller must send the contents of this file, encoded in hexadecimal. - - If the 'HashedControlPassword' option is set, it must contain the salted - hash of a secret password. The salted hash is computed according to the - S2K algorithm in RFC 2440 (OpenPGP), and prefixed with the s2k specifier. - This is then encoded in hexadecimal, prefixed by the indicator sequence - "16:". Thus, for example, the password 'foo' could encode to: - 16:660537E3E1CD49996044A3BF558097A981F539FEA2F9DA662B4626C1C2 - ++++++++++++++++**^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - salt hashed value - indicator - You can generate the salt of a password by calling - 'tor --hash-password ' - or by using the example code in the Python and Java controller libraries. - To authenticate under this scheme, the controller sends Tor the original - secret that was used to generate the password, either as a quoted string - or encoded in hexadecimal. - -5.2. Don't let the buffer get too big. - - If you ask for lots of events, and 16MB of them queue up on the buffer, - the Tor process will close the socket. - -5.3. Backward compatibility with v0 control protocol. - - The 'version 0' control protocol was replaced in Tor 0.1.1.x. Support - was removed in Tor 0.2.0.x. Every non-obsolete version of Tor now - supports the version 1 control protocol. - - For backward compatibility with the "version 0" control protocol, - Tor used to check whether the third octet of the first command is zero. - (If it was, Tor assumed that version 0 is in use.) - - This compatibility was removed in Tor 0.1.2.16 and 0.2.0.4-alpha. - -5.4. Tor config options for use by controllers - - Tor provides a few special configuration options for use by controllers. - These options can be set and examined by the SETCONF and GETCONF commands, - but are not saved to disk by SAVECONF. - - Generally, these options make Tor unusable by disabling a portion of Tor's - normal operations. Unless a controller provides replacement functionality - to fill this gap, Tor will not correctly handle user requests. - - __AllDirOptionsPrivate - - If true, Tor will try to launch all directory operations through - anonymous connections. (Ordinarily, Tor only tries to anonymize - requests related to hidden services.) This option will slow down - directory access, and may stop Tor from working entirely if it does not - yet have enough directory information to build circuits. - - (Boolean. Default: "0".) - - __DisablePredictedCircuits - - If true, Tor will not launch preemptive "general-purpose" circuits for - streams to attach to. (It will still launch circuits for testing and - for hidden services.) - - (Boolean. Default: "0".) - - __LeaveStreamsUnattached - - If true, Tor will not automatically attach new streams to circuits; - instead, the controller must attach them with ATTACHSTREAM. If the - controller does not attach the streams, their data will never be routed. - - (Boolean. Default: "0".) - - __HashedControlSessionPassword - - As HashedControlPassword, but is not saved to the torrc file by - SAVECONF. Added in Tor 0.2.0.20-rc. - - __ReloadTorrcOnSIGHUP - - If this option is true (the default), we reload the torrc from disk - every time we get a SIGHUP (from the controller or via a signal). - Otherwise, we don't. This option exists so that controllers can keep - their options from getting overwritten when a user sends Tor a HUP for - some other reason (for example, to rotate the logs). - - (Boolean. Default: "1") - -5.5. Phases from the Bootstrap status event. - - This section describes the various bootstrap phases currently reported - by Tor. Controllers should not assume that the percentages and tags - listed here will continue to match up, or even that the tags will stay - in the same order. Some phases might also be skipped (not reported) - if the associated bootstrap step is already complete, or if the phase - no longer is necessary. Only "starting" and "done" are guaranteed to - exist in all future versions. - - Current Tor versions enter these phases in order, monotonically. - Future Tors MAY revisit earlier stages. - - Phase 0: - tag=starting summary="Starting" - - Tor starts out in this phase. - - Phase 5: - tag=conn_dir summary="Connecting to directory mirror" - - Tor sends this event as soon as Tor has chosen a directory mirror -- - e.g. one of the authorities if bootstrapping for the first time or - after a long downtime, or one of the relays listed in its cached - directory information otherwise. - - Tor will stay at this phase until it has successfully established - a TCP connection with some directory mirror. Problems in this phase - generally happen because Tor doesn't have a network connection, or - because the local firewall is dropping SYN packets. - - Phase 10: - tag=handshake_dir summary="Finishing handshake with directory mirror" - - This event occurs when Tor establishes a TCP connection with a relay used - as a directory mirror (or its https proxy if it's using one). Tor remains - in this phase until the TLS handshake with the relay is finished. - - Problems in this phase generally happen because Tor's firewall is - doing more sophisticated MITM attacks on it, or doing packet-level - keyword recognition of Tor's handshake. - - Phase 15: - tag=onehop_create summary="Establishing one-hop circuit for dir info" - - Once TLS is finished with a relay, Tor will send a CREATE_FAST cell - to establish a one-hop circuit for retrieving directory information. - It will remain in this phase until it receives the CREATED_FAST cell - back, indicating that the circuit is ready. - - Phase 20: - tag=requesting_status summary="Asking for networkstatus consensus" - - Once we've finished our one-hop circuit, we will start a new stream - for fetching the networkstatus consensus. We'll stay in this phase - until we get the 'connected' relay cell back, indicating that we've - established a directory connection. - - Phase 25: - tag=loading_status summary="Loading networkstatus consensus" - - Once we've established a directory connection, we will start fetching - the networkstatus consensus document. This could take a while; this - phase is a good opportunity for using the "progress" keyword to indicate - partial progress. - - This phase could stall if the directory mirror we picked doesn't - have a copy of the networkstatus consensus so we have to ask another, - or it does give us a copy but we don't find it valid. - - Phase 40: - tag=loading_keys summary="Loading authority key certs" - - Sometimes when we've finished loading the networkstatus consensus, - we find that we don't have all the authority key certificates for the - keys that signed the consensus. At that point we put the consensus we - fetched on hold and fetch the keys so we can verify the signatures. - - Phase 45 - tag=requesting_descriptors summary="Asking for relay descriptors" - - Once we have a valid networkstatus consensus and we've checked all - its signatures, we start asking for relay descriptors. We stay in this - phase until we have received a 'connected' relay cell in response to - a request for descriptors. - - Phase 50: - tag=loading_descriptors summary="Loading relay descriptors" - - We will ask for relay descriptors from several different locations, - so this step will probably make up the bulk of the bootstrapping, - especially for users with slow connections. We stay in this phase until - we have descriptors for at least 1/4 of the usable relays listed in - the networkstatus consensus. This phase is also a good opportunity to - use the "progress" keyword to indicate partial steps. - - Phase 80: - tag=conn_or summary="Connecting to entry guard" - - Once we have a valid consensus and enough relay descriptors, we choose - some entry guards and start trying to build some circuits. This step - is similar to the "conn_dir" phase above; the only difference is - the context. - - If a Tor starts with enough recent cached directory information, - its first bootstrap status event will be for the conn_or phase. - - Phase 85: - tag=handshake_or summary="Finishing handshake with entry guard" - - This phase is similar to the "handshake_dir" phase, but it gets reached - if we finish a TCP connection to a Tor relay and we have already reached - the "conn_or" phase. We'll stay in this phase until we complete a TLS - handshake with a Tor relay. - - Phase 90: - tag=circuit_create summary="Establishing circuits" - - Once we've finished our TLS handshake with an entry guard, we will - set about trying to make some 3-hop circuits in case we need them soon. - - Phase 100: - tag=done summary="Done" - - A full 3-hop exit circuit has been established. Tor is ready to handle - application connections now. - diff --git a/doc/spec/dir-spec-v1.txt b/doc/spec/dir-spec-v1.txt deleted file mode 100644 index 286df664e..000000000 --- a/doc/spec/dir-spec-v1.txt +++ /dev/null @@ -1,315 +0,0 @@ -$Id$ - - Tor Protocol Specification - - Roger Dingledine - Nick Mathewson - -0. Preliminaries - - THIS SPECIFICATION IS OBSOLETE. - - This document specifies the Tor directory protocol as used in version - 0.1.0.x and earlier. See dir-spec.txt for a current version. - -1. Basic operation - - There is a small number of directory authorities, and a larger number of - caches. Client and servers know public keys for the directory authorities. - Tor servers periodically upload self-signed "router descriptors" to the - directory authorities. Each authority publishes a self-signed "directory" - (containing all the router descriptors it knows, and a statement on which - are running) and a self-signed "running routers" document containing only - the statement on which routers are running. - - All Tors periodically download these documents, downloading the directory - less frequently than they do the "running routers" document. Clients - preferentially download from caches rather than authorities. - -1.1. Document format - - Router descriptors, directories, and running-routers documents all obey the - following lightweight extensible information format. - - The highest level object is a Document, which consists of one or more - Items. Every Item begins with a KeywordLine, followed by one or more - Objects. A KeywordLine begins with a Keyword, optionally followed by - whitespace and more non-newline characters, and ends with a newline. A - Keyword is a sequence of one or more characters in the set [A-Za-z0-9-]. - An Object is a block of encoded data in pseudo-Open-PGP-style - armor. (cf. RFC 2440) - - More formally: - - Document ::= (Item | NL)+ - Item ::= KeywordLine Object* - KeywordLine ::= Keyword NL | Keyword WS ArgumentsChar+ NL - Keyword = KeywordChar+ - KeywordChar ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-' - ArgumentChar ::= any printing ASCII character except NL. - WS = (SP | TAB)+ - Object ::= BeginLine Base-64-encoded-data EndLine - BeginLine ::= "-----BEGIN " Keyword "-----" NL - EndLine ::= "-----END " Keyword "-----" NL - - The BeginLine and EndLine of an Object must use the same keyword. - - When interpreting a Document, software MUST reject any document containing a - KeywordLine that starts with a keyword it doesn't recognize. - - The "opt" keyword is reserved for non-critical future extensions. All - implementations MUST ignore any item of the form "opt keyword ....." when - they would not recognize "keyword ....."; and MUST treat "opt keyword ....." - as synonymous with "keyword ......" when keyword is recognized. - -2. Router descriptor format. - - Every router descriptor MUST start with a "router" Item; MUST end with a - "router-signature" Item and an extra NL; and MUST contain exactly one - instance of each of the following Items: "published" "onion-key" "link-key" - "signing-key" "bandwidth". Additionally, a router descriptor MAY contain - any number of "accept", "reject", "fingerprint", "uptime", and "opt" Items. - Other than "router" and "router-signature", the items may appear in any - order. - - The items' formats are as follows: - "router" nickname address ORPort SocksPort DirPort - - Indicates the beginning of a router descriptor. "address" - must be an IPv4 address in dotted-quad format. The last - three numbers indicate the TCP ports at which this OR exposes - functionality. ORPort is a port at which this OR accepts TLS - connections for the main OR protocol; SocksPort is deprecated and - should always be 0; and DirPort is the port at which this OR accepts - directory-related HTTP connections. If any port is not supported, - the value 0 is given instead of a port number. - - "bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed - - Estimated bandwidth for this router, in bytes per second. The - "average" bandwidth is the volume per second that the OR is willing - to sustain over long periods; the "burst" bandwidth is the volume - that the OR is willing to sustain in very short intervals. The - "observed" value is an estimate of the capacity this server can - handle. The server remembers the max bandwidth sustained output - over any ten second period in the past day, and another sustained - input. The "observed" value is the lesser of these two numbers. - - "platform" string - - A human-readable string describing the system on which this OR is - running. This MAY include the operating system, and SHOULD include - the name and version of the software implementing the Tor protocol. - - "published" YYYY-MM-DD HH:MM:SS - - The time, in GMT, when this descriptor was generated. - - "fingerprint" - - A fingerprint (a HASH_LEN-byte of asn1 encoded public key, encoded - in hex, with a single space after every 4 characters) for this router's - identity key. A descriptor is considered invalid (and MUST be - rejected) if the fingerprint line does not match the public key. - - [We didn't start parsing this line until Tor 0.1.0.6-rc; it should - be marked with "opt" until earlier versions of Tor are obsolete.] - - "hibernating" 0|1 - - If the value is 1, then the Tor server was hibernating when the - descriptor was published, and shouldn't be used to build circuits. - - [We didn't start parsing this line until Tor 0.1.0.6-rc; it should - be marked with "opt" until earlier versions of Tor are obsolete.] - - "uptime" - - The number of seconds that this OR process has been running. - - "onion-key" NL a public key in PEM format - - This key is used to encrypt EXTEND cells for this OR. The key MUST - be accepted for at least XXXX hours after any new key is published in - a subsequent descriptor. - - "signing-key" NL a public key in PEM format - - The OR's long-term identity key. - - "accept" exitpattern - "reject" exitpattern - - These lines, in order, describe the rules that an OR follows when - deciding whether to allow a new stream to a given address. The - 'exitpattern' syntax is described below. - - "router-signature" NL Signature NL - - The "SIGNATURE" object contains a signature of the PKCS1-padded - hash of the entire router descriptor, taken from the beginning of the - "router" line, through the newline after the "router-signature" line. - The router descriptor is invalid unless the signature is performed - with the router's identity key. - - "contact" info NL - - Describes a way to contact the server's administrator, preferably - including an email address and a PGP key fingerprint. - - "family" names NL - - 'Names' is a whitespace-separated list of server nicknames. If two ORs - list one another in their "family" entries, then OPs should treat them - as a single OR for the purpose of path selection. - - For example, if node A's descriptor contains "family B", and node B's - descriptor contains "family A", then node A and node B should never - be used on the same circuit. - - "read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - "write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - - Declare how much bandwidth the OR has used recently. Usage is divided - into intervals of NSEC seconds. The YYYY-MM-DD HH:MM:SS field defines - the end of the most recent interval. The numbers are the number of - bytes used in the most recent intervals, ordered from oldest to newest. - - [We didn't start parsing these lines until Tor 0.1.0.6-rc; they should - be marked with "opt" until earlier versions of Tor are obsolete.] - -2.1. Nonterminals in routerdescriptors - - nickname ::= between 1 and 19 alphanumeric characters, case-insensitive. - - exitpattern ::= addrspec ":" portspec - portspec ::= "*" | port | port "-" port - port ::= an integer between 1 and 65535, inclusive. - addrspec ::= "*" | ip4spec | ip6spec - ipv4spec ::= ip4 | ip4 "/" num_ip4_bits | ip4 "/" ip4mask - ip4 ::= an IPv4 address in dotted-quad format - ip4mask ::= an IPv4 mask in dotted-quad format - num_ip4_bits ::= an integer between 0 and 32 - ip6spec ::= ip6 | ip6 "/" num_ip6_bits - ip6 ::= an IPv6 address, surrounded by square brackets. - num_ip6_bits ::= an integer between 0 and 128 - - Ports are required; if they are not included in the router - line, they must appear in the "ports" lines. - -3. Directory format - - A Directory begins with a "signed-directory" item, followed by one each of - the following, in any order: "recommended-software", "published", - "router-status", "dir-signing-key". It may include any number of "opt" - items. After these items, a directory includes any number of router - descriptors, and a single "directory-signature" item. - - "signed-directory" - - Indicates the start of a directory. - - "published" YYYY-MM-DD HH:MM:SS - - The time at which this directory was generated and signed, in GMT. - - "dir-signing-key" - - The key used to sign this directory; see "signing-key" for format. - - "recommended-software" comma-separated-version-list - - A list of which versions of which implementations are currently - believed to be secure and compatible with the network. - - "running-routers" whitespace-separated-list - - A description of which routers are currently believed to be up or - down. Every entry consists of an optional "!", followed by either an - OR's nickname, or "$" followed by a hexadecimal encoding of the hash - of an OR's identity key. If the "!" is included, the router is - believed not to be running; otherwise, it is believed to be running. - If a router's nickname is given, exactly one router of that nickname - will appear in the directory, and that router is "approved" by the - directory server. If a hashed identity key is given, that OR is not - "approved". [XXXX The 'running-routers' line is only provided for - backward compatibility. New code should parse 'router-status' - instead.] - - "router-status" whitespace-separated-list - - A description of which routers are currently believed to be up or - down, and which are verified or unverified. Contains one entry for - every router that the directory server knows. Each entry is of the - format: - - !name=$digest [Verified router, currently not live.] - name=$digest [Verified router, currently live.] - !$digest [Unverified router, currently not live.] - or $digest [Unverified router, currently live.] - - (where 'name' is the router's nickname and 'digest' is a hexadecimal - encoding of the hash of the routers' identity key). - - When parsing this line, clients should only mark a router as - 'verified' if its nickname AND digest match the one provided. - - "directory-signature" nickname-of-dirserver NL Signature - - The signature is computed by computing the digest of the - directory, from the characters "signed-directory", through the newline - after "directory-signature". This digest is then padded with PKCS.1, - and signed with the directory server's signing key. - - If software encounters an unrecognized keyword in a single router descriptor, - it MUST reject only that router descriptor, and continue using the - others. Because this mechanism is used to add 'critical' extensions to - future versions of the router descriptor format, implementation should treat - it as a normal occurrence and not, for example, report it to the user as an - error. [Versions of Tor prior to 0.1.1 did this.] - - If software encounters an unrecognized keyword in the directory header, - it SHOULD reject the entire directory. - -4. Network-status descriptor - - A "network-status" (a.k.a "running-routers") document is a truncated - directory that contains only the current status of a list of nodes, not - their actual descriptors. It contains exactly one of each of the following - entries. - - "network-status" - - Must appear first. - - "published" YYYY-MM-DD HH:MM:SS - - (see section 3 above) - - "router-status" list - - (see section 3 above) - - "directory-signature" NL signature - - (see section 3 above) - -5. Behavior of a directory server - - lists nodes that are connected currently - speaks HTTP on a socket, spits out directory on request - - Directory servers listen on a certain port (the DirPort), and speak a - limited version of HTTP 1.0. Clients send either GET or POST commands. - The basic interactions are: - "%s %s HTTP/1.0\r\nContent-Length: %lu\r\nHost: %s\r\n\r\n", - command, url, content-length, host. - Get "/tor/" to fetch a full directory. - Get "/tor/dir.z" to fetch a compressed full directory. - Get "/tor/running-routers" to fetch a network-status descriptor. - Post "/tor/" to post a server descriptor, with the body of the - request containing the descriptor. - - "host" is used to specify the address:port of the dirserver, so - the request can survive going through HTTP proxies. - diff --git a/doc/spec/dir-spec-v2.txt b/doc/spec/dir-spec-v2.txt deleted file mode 100644 index 4873c4a72..000000000 --- a/doc/spec/dir-spec-v2.txt +++ /dev/null @@ -1,897 +0,0 @@ -$Id$ - - Tor directory protocol, version 2 - -0. Scope and preliminaries - - This directory protocol is used by Tor version 0.1.1.x and 0.1.2.x. See - dir-spec-v1.txt for information on earlier versions, and dir-spec.txt - for information on later versions. - -0.1. Goals and motivation - - There were several problems with the way Tor handles directory information - in version 0.1.0.x and earlier. Here are the problems we try to fix with - this new design, already implemented in 0.1.1.x: - 1. Directories were very large and use up a lot of bandwidth: clients - downloaded descriptors for all router several times an hour. - 2. Every directory authority was a trust bottleneck: if a single - directory authority lied, it could make clients believe for a time an - arbitrarily distorted view of the Tor network. - 3. Our current "verified server" system is kind of nonsensical. - - 4. Getting more directory authorities would add more points of failure - and worsen possible partitioning attacks. - - There are two problems that remain unaddressed by this design. - 5. Requiring every client to know about every router won't scale. - 6. Requiring every directory cache to know every router won't scale. - - We attempt to fix 1-4 here, and to build a solution that will work when we - figure out an answer for 5. We haven't thought at all about what to do - about 6. - -1. Outline - - There is a small set (say, around 10) of semi-trusted directory - authorities. A default list of authorities is shipped with the Tor - software. Users can change this list, but are encouraged not to do so, in - order to avoid partitioning attacks. - - Routers periodically upload signed "descriptors" to the directory - authorities describing their keys, capabilities, and other information. - Routers may act as directory mirrors (also called "caches"), to reduce - load on the directory authorities. They announce this in their - descriptors. - - Each directory authority periodically generates and signs a compact - "network status" document that lists that authority's view of the current - descriptors and status for known routers, but which does not include the - descriptors themselves. - - Directory mirrors download, cache, and re-serve network-status documents - to clients. - - Clients, directory mirrors, and directory authorities all use - network-status documents to find out when their list of routers is - out-of-date. If it is, they download any missing router descriptors. - Clients download missing descriptors from mirrors; mirrors and authorities - download from authorities. Descriptors are downloaded by the hash of the - descriptor, not by the server's identity key: this prevents servers from - attacking clients by giving them descriptors nobody else uses. - - All directory information is uploaded and downloaded with HTTP. - - Coordination among directory authorities is done client-side: clients - compute a vote-like algorithm among the network-status documents they - have, and base their decisions on the result. - -1.1. What's different from 0.1.0.x? - - Clients used to download a signed concatenated set of router descriptors - (called a "directory") from directory mirrors, regardless of which - descriptors had changed. - - Between downloading directories, clients would download "network-status" - documents that would list which servers were supposed to running. - - Clients would always believe the most recently published network-status - document they were served. - - Routers used to upload fresh descriptors all the time, whether their keys - and other information had changed or not. - -1.2. Document meta-format - - Router descriptors, directories, and running-routers documents all obey the - following lightweight extensible information format. - - The highest level object is a Document, which consists of one or more - Items. Every Item begins with a KeywordLine, followed by one or more - Objects. A KeywordLine begins with a Keyword, optionally followed by - whitespace and more non-newline characters, and ends with a newline. A - Keyword is a sequence of one or more characters in the set [A-Za-z0-9-]. - An Object is a block of encoded data in pseudo-Open-PGP-style - armor. (cf. RFC 2440) - - More formally: - - Document ::= (Item | NL)+ - Item ::= KeywordLine Object* - KeywordLine ::= Keyword NL | Keyword WS ArgumentsChar+ NL - Keyword = KeywordChar+ - KeywordChar ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-' - ArgumentChar ::= any printing ASCII character except NL. - WS = (SP | TAB)+ - Object ::= BeginLine Base-64-encoded-data EndLine - BeginLine ::= "-----BEGIN " Keyword "-----" NL - EndLine ::= "-----END " Keyword "-----" NL - - The BeginLine and EndLine of an Object must use the same keyword. - - When interpreting a Document, software MUST ignore any KeywordLine that - starts with a keyword it doesn't recognize; future implementations MUST NOT - require current clients to understand any KeywordLine not currently - described. - - The "opt" keyword was used until Tor 0.1.2.5-alpha for non-critical future - extensions. All implementations MUST ignore any item of the form "opt - keyword ....." when they would not recognize "keyword ....."; and MUST - treat "opt keyword ....." as synonymous with "keyword ......" when keyword - is recognized. - - Implementations before 0.1.2.5-alpha rejected any document with a - KeywordLine that started with a keyword that they didn't recognize. - Implementations MUST prefix items not recognized by older versions of Tor - with an "opt" until those versions of Tor are obsolete. - - Other implementations that want to extend Tor's directory format MAY - introduce their own items. The keywords for extension items SHOULD start - with the characters "x-" or "X-", to guarantee that they will not conflict - with keywords used by future versions of Tor. - -2. Router operation - - ORs SHOULD generate a new router descriptor whenever any of the - following events have occurred: - - - A period of time (18 hrs by default) has passed since the last - time a descriptor was generated. - - - A descriptor field other than bandwidth or uptime has changed. - - - Bandwidth has changed by at least a factor of 2 from the last time a - descriptor was generated, and at least a given interval of time - (20 mins by default) has passed since then. - - - Its uptime has been reset (by restarting). - - After generating a descriptor, ORs upload it to every directory - authority they know, by posting it to the URL - - http:///tor/ - -2.1. Router descriptor format - - Every router descriptor MUST start with a "router" Item; MUST end with a - "router-signature" Item and an extra NL; and MUST contain exactly one - instance of each of the following Items: "published" "onion-key" - "signing-key" "bandwidth". - - A router descriptor MAY have zero or one of each of the following Items, - but MUST NOT have more than one: "contact", "uptime", "fingerprint", - "hibernating", "read-history", "write-history", "eventdns", "platform", - "family". - - Additionally, a router descriptor MAY contain any number of "accept", - "reject", and "opt" Items. Other than "router" and "router-signature", - the items may appear in any order. - - The items' formats are as follows: - "router" nickname address ORPort SocksPort DirPort - - Indicates the beginning of a router descriptor. "address" must be an - IPv4 address in dotted-quad format. The last three numbers indicate - the TCP ports at which this OR exposes functionality. ORPort is a port - at which this OR accepts TLS connections for the main OR protocol; - SocksPort is deprecated and should always be 0; and DirPort is the - port at which this OR accepts directory-related HTTP connections. If - any port is not supported, the value 0 is given instead of a port - number. - - "bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed - - Estimated bandwidth for this router, in bytes per second. The - "average" bandwidth is the volume per second that the OR is willing to - sustain over long periods; the "burst" bandwidth is the volume that - the OR is willing to sustain in very short intervals. The "observed" - value is an estimate of the capacity this server can handle. The - server remembers the max bandwidth sustained output over any ten - second period in the past day, and another sustained input. The - "observed" value is the lesser of these two numbers. - - "platform" string - - A human-readable string describing the system on which this OR is - running. This MAY include the operating system, and SHOULD include - the name and version of the software implementing the Tor protocol. - - "published" YYYY-MM-DD HH:MM:SS - - The time, in GMT, when this descriptor was generated. - - "fingerprint" - - A fingerprint (a HASH_LEN-byte of asn1 encoded public key, encoded in - hex, with a single space after every 4 characters) for this router's - identity key. A descriptor is considered invalid (and MUST be - rejected) if the fingerprint line does not match the public key. - - [We didn't start parsing this line until Tor 0.1.0.6-rc; it should - be marked with "opt" until earlier versions of Tor are obsolete.] - - "hibernating" 0|1 - - If the value is 1, then the Tor server was hibernating when the - descriptor was published, and shouldn't be used to build circuits. - - [We didn't start parsing this line until Tor 0.1.0.6-rc; it should be - marked with "opt" until earlier versions of Tor are obsolete.] - - "uptime" - - The number of seconds that this OR process has been running. - - "onion-key" NL a public key in PEM format - - This key is used to encrypt EXTEND cells for this OR. The key MUST be - accepted for at least 1 week after any new key is published in a - subsequent descriptor. - - "signing-key" NL a public key in PEM format - - The OR's long-term identity key. - - "accept" exitpattern - "reject" exitpattern - - These lines describe the rules that an OR follows when - deciding whether to allow a new stream to a given address. The - 'exitpattern' syntax is described below. The rules are considered in - order; if no rule matches, the address will be accepted. For clarity, - the last such entry SHOULD be accept *:* or reject *:*. - - "router-signature" NL Signature NL - - The "SIGNATURE" object contains a signature of the PKCS1-padded - hash of the entire router descriptor, taken from the beginning of the - "router" line, through the newline after the "router-signature" line. - The router descriptor is invalid unless the signature is performed - with the router's identity key. - - "contact" info NL - - Describes a way to contact the server's administrator, preferably - including an email address and a PGP key fingerprint. - - "family" names NL - - 'Names' is a space-separated list of server nicknames or - hexdigests. If two ORs list one another in their "family" entries, - then OPs should treat them as a single OR for the purpose of path - selection. - - For example, if node A's descriptor contains "family B", and node B's - descriptor contains "family A", then node A and node B should never - be used on the same circuit. - - "read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - "write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - - Declare how much bandwidth the OR has used recently. Usage is divided - into intervals of NSEC seconds. The YYYY-MM-DD HH:MM:SS field - defines the end of the most recent interval. The numbers are the - number of bytes used in the most recent intervals, ordered from - oldest to newest. - - [We didn't start parsing these lines until Tor 0.1.0.6-rc; they should - be marked with "opt" until earlier versions of Tor are obsolete.] - - "eventdns" bool NL - - Declare whether this version of Tor is using the newer enhanced - dns logic. Versions of Tor without eventdns SHOULD NOT be used for - reverse hostname lookups. - - [All versions of Tor before 0.1.2.2-alpha should be assumed to have - this option set to 0 if it is not present. All Tor versions at - 0.1.2.2-alpha or later should be assumed to have this option set to - 1 if it is not present. Until 0.1.2.1-alpha-dev, this option was - not generated, even when eventdns was in use. Versions of Tor - before 0.1.2.1-alpha-dev did not parse this option, so it should be - marked "opt". With 0.2.0.1-alpha, the old 'dnsworker' logic has - been removed, rendering this option of historical interest only.] - -2.2. Nonterminals in router descriptors - - nickname ::= between 1 and 19 alphanumeric characters, case-insensitive. - hexdigest ::= a '$', followed by 20 hexadecimal characters. - [Represents a server by the digest of its identity key.] - - exitpattern ::= addrspec ":" portspec - portspec ::= "*" | port | port "-" port - port ::= an integer between 1 and 65535, inclusive. - [Some implementations incorrectly generate ports with value 0. - Implementations SHOULD accept this, and SHOULD NOT generate it.] - - addrspec ::= "*" | ip4spec | ip6spec - ipv4spec ::= ip4 | ip4 "/" num_ip4_bits | ip4 "/" ip4mask - ip4 ::= an IPv4 address in dotted-quad format - ip4mask ::= an IPv4 mask in dotted-quad format - num_ip4_bits ::= an integer between 0 and 32 - ip6spec ::= ip6 | ip6 "/" num_ip6_bits - ip6 ::= an IPv6 address, surrounded by square brackets. - num_ip6_bits ::= an integer between 0 and 128 - - bool ::= "0" | "1" - - Ports are required; if they are not included in the router - line, they must appear in the "ports" lines. - -3. Network status format - - Directory authorities generate, sign, and compress network-status - documents. Directory servers SHOULD generate a fresh network-status - document when the contents of such a document would be different from the - last one generated, and some time (at least one second, possibly longer) - has passed since the last one was generated. - - The network status document contains a preamble, a set of router status - entries, and a signature, in that order. - - We use the same meta-format as used for directories and router descriptors - in "tor-spec.txt". Implementations MAY insert blank lines - for clarity between sections; these blank lines are ignored. - Implementations MUST NOT depend on blank lines in any particular location. - - As used here, "whitespace" is a sequence of 1 or more tab or space - characters. - - The preamble contains: - - "network-status-version" -- A document format version. For this - specification, the version is "2". - "dir-source" -- The authority's hostname, current IP address, and - directory port, all separated by whitespace. - "fingerprint" -- A base16-encoded hash of the signing key's - fingerprint, with no additional spaces added. - "contact" -- An arbitrary string describing how to contact the - directory server's administrator. Administrators should include at - least an email address and a PGP fingerprint. - "dir-signing-key" -- The directory server's public signing key. - "client-versions" -- A comma-separated list of recommended client - versions. - "server-versions" -- A comma-separated list of recommended server - versions. - "published" -- The publication time for this network-status object. - "dir-options" -- A set of flags, in any order, separated by whitespace: - "Names" if this directory authority performs name bindings. - "Versions" if this directory authority recommends software versions. - "BadExits" if the directory authority flags nodes that it believes - are performing incorrectly as exit nodes. - "BadDirectories" if the directory authority flags nodes that it - believes are performing incorrectly as directory caches. - - The dir-options entry is optional. The "-versions" entries are required if - the "Versions" flag is present. The other entries are required and must - appear exactly once. The "network-status-version" entry must appear first; - the others may appear in any order. Implementations MUST ignore - additional arguments to the items above, and MUST ignore unrecognized - flags. - - For each router, the router entry contains: (This format is designed for - conciseness.) - - "r" -- followed by the following elements, in order, separated by - whitespace: - - The OR's nickname, - - A hash of its identity key, encoded in base64, with trailing = - signs removed. - - A hash of its most recent descriptor, encoded in base64, with - trailing = signs removed. (The hash is calculated as for - computing the signature of a descriptor.) - - The publication time of its most recent descriptor, in the form - YYYY-MM-DD HH:MM:SS, in GMT. - - An IP address - - An OR port - - A directory port (or "0" for none") - "s" -- A series of whitespace-separated status flags, in any order: - "Authority" if the router is a directory authority. - "BadExit" if the router is believed to be useless as an exit node - (because its ISP censors it, because it is behind a restrictive - proxy, or for some similar reason). - "BadDirectory" if the router is believed to be useless as a - directory cache (because its directory port isn't working, - its bandwidth is always throttled, or for some similar - reason). - "Exit" if the router is useful for building general-purpose exit - circuits. - "Fast" if the router is suitable for high-bandwidth circuits. - "Guard" if the router is suitable for use as an entry guard. - "Named" if the router's identity-nickname mapping is canonical, - and this authority binds names. - "Stable" if the router is suitable for long-lived circuits. - "Running" if the router is currently usable. - "Valid" if the router has been 'validated'. - "V2Dir" if the router implements this protocol. - "v" -- The version of the Tor protocol that this server is running. If - the value begins with "Tor" SP, the rest of the string is a Tor - version number, and the protocol is "The Tor protocol as supported - by the given version of Tor." Otherwise, if the value begins with - some other string, Tor has upgraded to a more sophisticated - protocol versioning system, and the protocol is "a version of the - Tor protocol more recent than any we recognize." - - The "r" entry for each router must appear first and is required. The - "s" entry is optional (see Section 3.1 below for how the flags are - decided). Unrecognized flags on the "s" line and extra elements - on the "r" line must be ignored. The "v" line is optional; it was not - supported until 0.1.2.5-alpha, and it must be preceded with an "opt" - until all earlier versions of Tor are obsolete. - - The signature section contains: - - "directory-signature" nickname-of-dirserver NL Signature - - Signature is a signature of this network-status document - (the document up until the signature, including the line - "directory-signature \n"), using the directory authority's - signing key. - - We compress the network status list with zlib before transmitting it. - -3.1. Establishing server status - - (This section describes how directory authorities choose which status - flags to apply to routers, as of Tor 0.1.1.18-rc. Later directory - authorities MAY do things differently, so long as clients keep working - well. Clients MUST NOT depend on the exact behaviors in this section.) - - In the below definitions, a router is considered "active" if it is - running, valid, and not hibernating. - - "Valid" -- a router is 'Valid' if it is running a version of Tor not - known to be broken, and the directory authority has not blacklisted - it as suspicious. - - "Named" -- Directory authority administrators may decide to support name - binding. If they do, then they must maintain a file of - nickname-to-identity-key mappings, and try to keep this file consistent - with other directory authorities. If they don't, they act as clients, and - report bindings made by other directory authorities (name X is bound to - identity Y if at least one binding directory lists it, and no directory - binds X to some other Y'.) A router is called 'Named' if the router - believes the given name should be bound to the given key. - - "Running" -- A router is 'Running' if the authority managed to connect to - it successfully within the last 30 minutes. - - "Stable" -- A router is 'Stable' if it is active, and either its - uptime is at least the median uptime for known active routers, or - its uptime is at least 30 days. Routers are never called stable if - they are running a version of Tor known to drop circuits stupidly. - (0.1.1.10-alpha through 0.1.1.16-rc are stupid this way.) - - "Fast" -- A router is 'Fast' if it is active, and its bandwidth is - in the top 7/8ths for known active routers. - - "Guard" -- A router is a possible 'Guard' if it is 'Stable' and its - bandwidth is above median for known active routers. If the total - bandwidth of active non-BadExit Exit servers is less than one third - of the total bandwidth of all active servers, no Exit is listed as - a Guard. - - "Authority" -- A router is called an 'Authority' if the authority - generating the network-status document believes it is an authority. - - "V2Dir" -- A router supports the v2 directory protocol if it has an open - directory port, and it is running a version of the directory protocol that - supports the functionality clients need. (Currently, this is - 0.1.1.9-alpha or later.) - - Directory server administrators may label some servers or IPs as - blacklisted, and elect not to include them in their network-status lists. - - Authorities SHOULD 'disable' any servers in excess of 3 on any single IP. - When there are more than 3 to choose from, authorities should first prefer - authorities to non-authorities, then prefer Running to non-Running, and - then prefer high-bandwidth to low-bandwidth. To 'disable' a server, the - authority *should* advertise it without the Running or Valid flag. - - Thus, the network-status list includes all non-blacklisted, - non-expired, non-superseded descriptors. - -4. Directory server operation - - All directory authorities and directory mirrors ("directory servers") - implement this section, except as noted. - -4.1. Accepting uploads (authorities only) - - When a router posts a signed descriptor to a directory authority, the - authority first checks whether it is well-formed and correctly - self-signed. If it is, the authority next verifies that the nickname - in question is not already assigned to a router with a different - public key. - Finally, the authority MAY check that the router is not blacklisted - because of its key, IP, or another reason. - - If the descriptor passes these tests, and the authority does not already - have a descriptor for a router with this public key, it accepts the - descriptor and remembers it. - - If the authority _does_ have a descriptor with the same public key, the - newly uploaded descriptor is remembered if its publication time is more - recent than the most recent old descriptor for that router, and either: - - There are non-cosmetic differences between the old descriptor and the - new one. - - Enough time has passed between the descriptors' publication times. - (Currently, 12 hours.) - - Differences between router descriptors are "non-cosmetic" if they would be - sufficient to force an upload as described in section 2 above. - - Note that the "cosmetic difference" test only applies to uploaded - descriptors, not to descriptors that the authority downloads from other - authorities. - -4.2. Downloading network-status documents (authorities and caches) - - All directory servers (authorities and mirrors) try to keep a fresh - set of network-status documents from every authority. To do so, - every 5 minutes, each authority asks every other authority for its - most recent network-status document. Every 15 minutes, each mirror - picks a random authority and asks it for the most recent network-status - documents for all the authorities the authority knows about (including - the chosen authority itself). - - Directory servers and mirrors remember and serve the most recent - network-status document they have from each authority. Other - network-status documents don't need to be stored. If the most recent - network-status document is over 10 days old, it is discarded anyway. - Mirrors SHOULD store and serve network-status documents from authorities - they don't recognize, but SHOULD NOT use such documents for any other - purpose. Mirrors SHOULD discard network-status documents older than 48 - hours. - -4.3. Downloading and storing router descriptors (authorities and caches) - - Periodically (currently, every 10 seconds), directory servers check - whether there are any specific descriptors (as identified by descriptor - hash in a network-status document) that they do not have and that they - are not currently trying to download. - - If so, the directory server launches requests to the authorities for these - descriptors, such that each authority is only asked for descriptors listed - in its most recent network-status. When more than one authority lists the - descriptor, we choose which to ask at random. - - If one of these downloads fails, we do not try to download that descriptor - from the authority that failed to serve it again unless we receive a newer - network-status from that authority that lists the same descriptor. - - Directory servers must potentially cache multiple descriptors for each - router. Servers must not discard any descriptor listed by any current - network-status document from any authority. If there is enough space to - store additional descriptors, servers SHOULD try to hold those which - clients are likely to download the most. (Currently, this is judged - based on the interval for which each descriptor seemed newest.) - - Authorities SHOULD NOT download descriptors for routers that they would - immediately reject for reasons listed in 3.1. - -4.4. HTTP URLs - - "Fingerprints" in these URLs are base-16-encoded SHA1 hashes. - - The authoritative network-status published by a host should be available at: - http:///tor/status/authority.z - - The network-status published by a host with fingerprint - should be available at: - http:///tor/status/fp/.z - - The network-status documents published by hosts with fingerprints - ,, should be available at: - http:///tor/status/fp/++.z - - The most recent network-status documents from all known authorities, - concatenated, should be available at: - http:///tor/status/all.z - - The most recent descriptor for a server whose identity key has a - fingerprint of should be available at: - http:///tor/server/fp/.z - - The most recent descriptors for servers with identity fingerprints - ,, should be available at: - http:///tor/server/fp/++.z - - (NOTE: Implementations SHOULD NOT download descriptors by identity key - fingerprint. This allows a corrupted server (in collusion with a cache) to - provide a unique descriptor to a client, and thereby partition that client - from the rest of the network.) - - The server descriptor with (descriptor) digest (in hex) should be - available at: - http:///tor/server/d/.z - - The most recent descriptors with digests ,, should be - available at: - http:///tor/server/d/++.z - - The most recent descriptor for this server should be at: - http:///tor/server/authority.z - [Nothing in the Tor protocol uses this resource yet, but it is useful - for debugging purposes. Also, the official Tor implementations - (starting at 0.1.1.x) use this resource to test whether a server's - own DirPort is reachable.] - - A concatenated set of the most recent descriptors for all known servers - should be available at: - http:///tor/server/all.z - - For debugging, directories SHOULD expose non-compressed objects at URLs like - the above, but without the final ".z". - Clients MUST handle compressed concatenated information in two forms: - - A concatenated list of zlib-compressed objects. - - A zlib-compressed concatenated list of objects. - Directory servers MAY generate either format: the former requires less - CPU, but the latter requires less bandwidth. - - Clients SHOULD use upper case letters (A-F) when base16-encoding - fingerprints. Servers MUST accept both upper and lower case fingerprints - in requests. - -5. Client operation: downloading information - - Every Tor that is not a directory server (that is, those that do - not have a DirPort set) implements this section. - -5.1. Downloading network-status documents - - Each client maintains an ordered list of directory authorities. - Insofar as possible, clients SHOULD all use the same ordered list. - - For each network-status document a client has, it keeps track of its - publication time *and* the time when the client retrieved it. Clients - consider a network-status document "live" if it was published within the - last 24 hours. - - Clients try to have a live network-status document hours from *every* - authority, and try to periodically get new network-status documents from - each authority in rotation as follows: - - If a client is missing a live network-status document for any - authority, it tries to fetch it from a directory cache. On failure, - the client waits briefly, then tries that network-status document - again from another cache. The client does not build circuits until it - has live network-status documents from more than half the authorities - it trusts, and it has descriptors for more than 1/4 of the routers - that it believes are running. - - If the most recently _retrieved_ network-status document is over 30 - minutes old, the client attempts to download a network-status document. - When choosing which documents to download, clients treat their list of - directory authorities as a circular ring, and begin with the authority - appearing immediately after the authority for their most recently - retrieved network-status document. If this attempt fails (either it - fails to download at all, or the one it gets is not as good as the - one it has), the client retries at other caches several times, before - moving on to the next network-status document in sequence. - - Clients discard all network-status documents over 24 hours old. - - If enough mirrors (currently 4) claim not to have a given network status, - we stop trying to download that authority's network-status, until we - download a new network-status that makes us believe that the authority in - question is running. Clients should wait a little longer after each - failure. - - Clients SHOULD try to batch as many network-status requests as possible - into each HTTP GET. - - (Note: clients can and should pick caches based on the network-status - information they have: once they have first fetched network-status info - from an authority, they should not need to go to the authority directly - again.) - -5.2. Downloading and storing router descriptors - - Clients try to have the best descriptor for each router. A descriptor is - "best" if: - * It is the most recently published descriptor listed for that router - by at least two network-status documents. - OR, - * No descriptor for that router is listed by two or more - network-status documents, and it is the most recently published - descriptor listed by any network-status document. - - Periodically (currently every 10 seconds) clients check whether there are - any "downloadable" descriptors. A descriptor is downloadable if: - - It is the "best" descriptor for some router. - - The descriptor was published at least 10 minutes in the past. - (This prevents clients from trying to fetch descriptors that the - mirrors have probably not yet retrieved and cached.) - - The client does not currently have it. - - The client is not currently trying to download it. - - The client would not discard it immediately upon receiving it. - - The client thinks it is running and valid (see 6.1 below). - - If at least 16 known routers have downloadable descriptors, or if - enough time (currently 10 minutes) has passed since the last time the - client tried to download descriptors, it launches requests for all - downloadable descriptors, as described in 5.3 below. - - When a descriptor download fails, the client notes it, and does not - consider the descriptor downloadable again until a certain amount of time - has passed. (Currently 0 seconds for the first failure, 60 seconds for the - second, 5 minutes for the third, 10 minutes for the fourth, and 1 day - thereafter.) Periodically (currently once an hour) clients reset the - failure count. - - No descriptors are downloaded until the client has downloaded more than - half of the network-status documents. - - Clients retain the most recent descriptor they have downloaded for each - router so long as it is not too old (currently, 48 hours), OR so long as - it is recommended by at least one networkstatus AND no "better" - descriptor has been downloaded. [Versions of Tor before 0.1.2.3-alpha - would discard descriptors simply for being published too far in the past.] - [The code seems to discard descriptors in all cases after they're 5 - days old. True? -RD] - -5.3. Managing downloads - - When a client has no live network-status documents, it downloads - network-status documents from a randomly chosen authority. In all other - cases, the client downloads from mirrors randomly chosen from among those - believed to be V2 directory servers. (This information comes from the - network-status documents; see 6 below.) - - When downloading multiple router descriptors, the client chooses multiple - mirrors so that: - - At least 3 different mirrors are used, except when this would result - in more than one request for under 4 descriptors. - - No more than 128 descriptors are requested from a single mirror. - - Otherwise, as few mirrors as possible are used. - After choosing mirrors, the client divides the descriptors among them - randomly. - - After receiving any response client MUST discard any network-status - documents and descriptors that it did not request. - -6. Using directory information - - Everyone besides directory authorities uses the approaches in this section - to decide which servers to use and what their keys are likely to be. - (Directory authorities just believe their own opinions, as in 3.1 above.) - -6.1. Choosing routers for circuits. - - Tor implementations only pay attention to "live" network-status documents. - A network status is "live" if it is the most recently downloaded network - status document for a given directory server, and the server is a - directory server trusted by the client, and the network-status document is - no more than 1 day old. - - For time-sensitive information, Tor implementations focus on "recent" - network-status documents. A network status is "recent" if it is live, and - if it was published in the last 60 minutes. If there are fewer - than 3 such documents, the most recently published 3 are "recent." If - there are fewer than 3 in all, all are "recent.") - - Circuits SHOULD NOT be built until the client has enough directory - information: network-statuses (or failed attempts to download - network-statuses) for all authorities, network-statuses for at more than - half of the authorities, and descriptors for at least 1/4 of the servers - believed to be running. - - A server is "listed" if it is included by more than half of the live - network status documents. Clients SHOULD NOT use unlisted servers. - - Clients believe the flags "Valid", "Exit", "Fast", "Guard", "Stable", and - "V2Dir" about a given router when they are asserted by more than half of - the live network-status documents. Clients believe the flag "Running" if - it is listed by more than half of the recent network-status documents. - - These flags are used as follows: - - - Clients SHOULD NOT use non-'Valid' or non-'Running' routers unless - requested to do so. - - - Clients SHOULD NOT use non-'Fast' routers for any purpose other than - very-low-bandwidth circuits (such as introduction circuits). - - - Clients SHOULD NOT use non-'Stable' routers for circuits that are - likely to need to be open for a very long time (such as those used for - IRC or SSH connections). - - - Clients SHOULD NOT choose non-'Guard' nodes when picking entry guard - nodes. - - - Clients SHOULD NOT download directory information from non-'V2Dir' - caches. - -6.2. Managing naming - - In order to provide human-memorable names for individual server - identities, some directory servers bind names to IDs. Clients handle - names in two ways: - - When a client encounters a name it has not mapped before: - - If all the live "Naming" network-status documents the client has - claim that the name binds to some identity ID, and the client has at - least three live network-status documents, the client maps the name to - ID. - - When a user tries to refer to a router with a name that does not have a - mapping under the above rules, the implementation SHOULD warn the user. - After giving the warning, the implementation MAY use a router that at - least one Naming authority maps the name to, so long as no other naming - authority maps that name to a different router. If no Naming authority - maps the name to a router, the implementation MAY use any router that - advertises the name. - - Not every router needs a nickname. When a router doesn't configure a - nickname, it publishes with the default nickname "Unnamed". Authorities - SHOULD NOT ever mark a router with this nickname as Named; client software - SHOULD NOT ever use a router in response to a user request for a router - called "Unnamed". - -6.3. Software versions - - An implementation of Tor SHOULD warn when it has fetched (or has - attempted to fetch and failed four consecutive times) a network-status - for each authority, and it is running a software version - not listed on more than half of the live "Versioning" network-status - documents. - -6.4. Warning about a router's status. - - If a router tries to publish its descriptor to a Naming authority - that has its nickname mapped to another key, the router SHOULD - warn the operator that it is either using the wrong key or is using - an already claimed nickname. - - If a router has fetched (or attempted to fetch and failed four - consecutive times) a network-status for every authority, and at - least one of the authorities is "Naming", and no live "Naming" - authorities publish a binding for the router's nickname, the - router MAY remind the operator that the chosen nickname is not - bound to this key at the authorities, and suggest contacting the - authority operators. - - ... - -6.5. Router protocol versions - - A client should believe that a router supports a given feature if that - feature is supported by the router or protocol versions in more than half - of the live networkstatus's "v" entries for that router. In other words, - if the "v" entries for some router are: - v Tor 0.0.8pre1 (from authority 1) - v Tor 0.1.2.11 (from authority 2) - v FutureProtocolDescription 99 (from authority 3) - then the client should believe that the router supports any feature - supported by 0.1.2.11. - - This is currently equivalent to believing the median declared version for - a router in all live networkstatuses. - -7. Standards compliance - - All clients and servers MUST support HTTP 1.0. - -7.1. HTTP headers - - Servers MAY set the Content-Length: header. Servers SHOULD set - Content-Encoding to "deflate" or "identity". - - Servers MAY include an X-Your-Address-Is: header, whose value is the - apparent IP address of the client connecting to them (as a dotted quad). - For directory connections tunneled over a BEGIN_DIR stream, servers SHOULD - report the IP from which the circuit carrying the BEGIN_DIR stream reached - them. [Servers before version 0.1.2.5-alpha reported 127.0.0.1 for all - BEGIN_DIR-tunneled connections.] - - Servers SHOULD disable caching of multiple network statuses or multiple - router descriptors. Servers MAY enable caching of single descriptors, - single network statuses, the list of all router descriptors, a v1 - directory, or a v1 running routers document. XXX mention times. - -7.2. HTTP status codes - - XXX We should write down what return codes dirservers send in what situations. - diff --git a/doc/spec/dir-spec.txt b/doc/spec/dir-spec.txt deleted file mode 100644 index 9a2a62bc4..000000000 --- a/doc/spec/dir-spec.txt +++ /dev/null @@ -1,1898 +0,0 @@ -$Id$ - - Tor directory protocol, version 3 - -0. Scope and preliminaries - - This directory protocol is used by Tor version 0.2.0.x-alpha and later. - See dir-spec-v1.txt for information on the protocol used up to the - 0.1.0.x series, and dir-spec-v2.txt for information on the protocol - used by the 0.1.1.x and 0.1.2.x series. - - Caches and authorities must still support older versions of the - directory protocols, until the versions of Tor that require them are - finally out of commission. See Section XXXX on backward compatibility. - - This document merges and supersedes the following proposals: - - 101 Voting on the Tor Directory System - 103 Splitting identity key from regularly used signing key - 104 Long and Short Router Descriptors - - AS OF 14 JUNE 2007, THIS SPECIFICATION HAS NOT YET BEEN COMPLETELY - IMPLEMENTED, OR COMPLETELY COMPLETED. - - XXX when to download certificates. - XXX timeline - XXX fill in XXXXs - -0.1. History - - The earliest versions of Onion Routing shipped with a list of known - routers and their keys. When the set of routers changed, users needed to - fetch a new list. - - The Version 1 Directory protocol - -------------------------------- - - Early versions of Tor (0.0.2) introduced "Directory authorities": servers - that served signed "directory" documents containing a list of signed - "router descriptors", along with short summary of the status of each - router. Thus, clients could get up-to-date information on the state of - the network automatically, and be certain that the list they were getting - was attested by a trusted directory authority. - - Later versions (0.0.8) added directory caches, which download - directories from the authorities and serve them to clients. Non-caches - fetch from the caches in preference to fetching from the authorities, thus - distributing bandwidth requirements. - - Also added during the version 1 directory protocol were "router status" - documents: short documents that listed only the up/down status of the - routers on the network, rather than a complete list of all the - descriptors. Clients and caches would fetch these documents far more - frequently than they would fetch full directories. - - The Version 2 Directory Protocol - -------------------------------- - - During the Tor 0.1.1.x series, Tor revised its handling of directory - documents in order to address two major problems: - - * Directories had grown quite large (over 1MB), and most directory - downloads consisted mainly of router descriptors that clients - already had. - - * Every directory authority was a trust bottleneck: if a single - directory authority lied, it could make clients believe for a time - an arbitrarily distorted view of the Tor network. (Clients - trusted the most recent signed document they downloaded.) Thus, - adding more authorities would make the system less secure, not - more. - - To address these, we extended the directory protocol so that - authorities now published signed "network status" documents. Each - network status listed, for every router in the network: a hash of its - identity key, a hash of its most recent descriptor, and a summary of - what the authority believed about its status. Clients would download - the authorities' network status documents in turn, and believe - statements about routers iff they were attested to by more than half of - the authorities. - - Instead of downloading all router descriptors at once, clients - downloaded only the descriptors that they did not have. Descriptors - were indexed by their digests, in order to prevent malicious caches - from giving different versions of a router descriptor to different - clients. - - Routers began working harder to upload new descriptors only when their - contents were substantially changed. - - -0.2. Goals of the version 3 protocol - - Version 3 of the Tor directory protocol tries to solve the following - issues: - - * A great deal of bandwidth used to transmit router descriptors was - used by two fields that are not actually used by Tor routers - (namely read-history and write-history). We save about 60% by - moving them into a separate document that most clients do not - fetch or use. - - * It was possible under certain perverse circumstances for clients - to download an unusual set of network status documents, thus - partitioning themselves from clients who have a more recent and/or - typical set of documents. Even under the best of circumstances, - clients were sensitive to the ages of the network status documents - they downloaded. Therefore, instead of having the clients - correlate multiple network status documents, we have the - authorities collectively vote on a single consensus network status - document. - - * The most sensitive data in the entire network (the identity keys - of the directory authorities) needed to be stored unencrypted so - that the authorities can sign network-status documents on the fly. - Now, the authorities' identity keys are stored offline, and used - to certify medium-term signing keys that can be rotated. - -0.3. Some Remaining questions - - Things we could solve on a v3 timeframe: - - The SHA-1 hash is showing its age. We should do something about our - dependency on it. We could probably future-proof ourselves here in - this revision, at least so far as documents from the authorities are - concerned. - - Too many things about the authorities are hardcoded by IP. - - Perhaps we should start accepting longer identity keys for routers - too. - - Things to solve eventually: - - Requiring every client to know about every router won't scale forever. - - Requiring every directory cache to know every router won't scale - forever. - - -1. Outline - - There is a small set (say, around 5-10) of semi-trusted directory - authorities. A default list of authorities is shipped with the Tor - software. Users can change this list, but are encouraged not to do so, - in order to avoid partitioning attacks. - - Every authority has a very-secret, long-term "Authority Identity Key". - This is stored encrypted and/or offline, and is used to sign "key - certificate" documents. Every key certificate contains a medium-term - (3-12 months) "authority signing key", that is used by the authority to - sign other directory information. (Note that the authority identity - key is distinct from the router identity key that the authority uses - in its role as an ordinary router.) - - Routers periodically upload signed "routers descriptors" to the - directory authorities describing their keys, capabilities, and other - information. Routers may also upload signed "extra info documents" - containing information that is not required for the Tor protocol. - Directory authorities serve router descriptors indexed by router - identity, or by hash of the descriptor. - - Routers may act as directory caches to reduce load on the directory - authorities. They announce this in their descriptors. - - Periodically, each directory authority generates a view of - the current descriptors and status for known routers. They send a - signed summary of this view (a "status vote") to the other - authorities. The authorities compute the result of this vote, and sign - a "consensus status" document containing the result of the vote. - - Directory caches download, cache, and re-serve consensus documents. - - Clients, directory caches, and directory authorities all use consensus - documents to find out when their list of routers is out-of-date. - (Directory authorities also use vote statuses.) If it is, they download - any missing router descriptors. Clients download missing descriptors - from caches; caches and authorities download from authorities. - Descriptors are downloaded by the hash of the descriptor, not by the - server's identity key: this prevents servers from attacking clients by - giving them descriptors nobody else uses. - - All directory information is uploaded and downloaded with HTTP. - - [Authorities also generate and caches also cache documents produced and - used by earlier versions of this protocol; see section XXX for notes.] - -1.1. What's different from version 2? - - Clients used to download multiple network status documents, - corresponding roughly to "status votes" above. They would compute the - result of the vote on the client side. - - Authorities used to sign documents using the same private keys they used - for their roles as routers. This forced them to keep these extremely - sensitive keys in memory unencrypted. - - All of the information in extra-info documents used to be kept in the - main descriptors. - -1.2. Document meta-format - - Router descriptors, directories, and running-routers documents all obey the - following lightweight extensible information format. - - The highest level object is a Document, which consists of one or more - Items. Every Item begins with a KeywordLine, followed by zero or more - Objects. A KeywordLine begins with a Keyword, optionally followed by - whitespace and more non-newline characters, and ends with a newline. A - Keyword is a sequence of one or more characters in the set [A-Za-z0-9-]. - An Object is a block of encoded data in pseudo-Open-PGP-style - armor. (cf. RFC 2440) - - More formally: - - NL = The ascii LF character (hex value 0x0a). - Document ::= (Item | NL)+ - Item ::= KeywordLine Object* - KeywordLine ::= Keyword NL | Keyword WS ArgumentChar+ NL - Keyword = KeywordChar+ - KeywordChar ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-' - ArgumentChar ::= any printing ASCII character except NL. - WS = (SP | TAB)+ - Object ::= BeginLine Base-64-encoded-data EndLine - BeginLine ::= "-----BEGIN " Keyword "-----" NL - EndLine ::= "-----END " Keyword "-----" NL - - The BeginLine and EndLine of an Object must use the same keyword. - - When interpreting a Document, software MUST ignore any KeywordLine that - starts with a keyword it doesn't recognize; future implementations MUST NOT - require current clients to understand any KeywordLine not currently - described. - - The "opt" keyword was used until Tor 0.1.2.5-alpha for non-critical future - extensions. All implementations MUST ignore any item of the form "opt - keyword ....." when they would not recognize "keyword ....."; and MUST - treat "opt keyword ....." as synonymous with "keyword ......" when keyword - is recognized. - - Implementations before 0.1.2.5-alpha rejected any document with a - KeywordLine that started with a keyword that they didn't recognize. - When generating documents that need to be read by older versions of Tor, - implementations MUST prefix items not recognized by older versions of - Tor with an "opt" until those versions of Tor are obsolete. [Note that - key certificates, status vote documents, extra info documents, and - status consensus documents will never be read by older versions of Tor.] - - Other implementations that want to extend Tor's directory format MAY - introduce their own items. The keywords for extension items SHOULD start - with the characters "x-" or "X-", to guarantee that they will not conflict - with keywords used by future versions of Tor. - - In our document descriptions below, we tag Items with a multiplicity in - brackets. Possible tags are: - - "At start, exactly once": These items MUST occur in every instance of - the document type, and MUST appear exactly once, and MUST be the - first item in their documents. - - "Exactly once": These items MUST occur exactly one time in every - instance of the document type. - - "At end, exactly once": These items MUST occur in every instance of - the document type, and MUST appear exactly once, and MUST be the - last item in their documents. - - "At most once": These items MAY occur zero or one times in any - instance of the document type, but MUST NOT occur more than once. - - "Any number": These items MAY occur zero, one, or more times in any - instance of the document type. - - "Once or more": These items MUST occur at least once in any instance - of the document type, and MAY occur more. - -1.3. Signing documents - - Every signable document below is signed in a similar manner, using a - given "Initial Item", a final "Signature Item", a digest algorithm, and - a signing key. - - The Initial Item must be the first item in the document. - - The Signature Item has the following format: - - [arguments] NL SIGNATURE NL - - The "SIGNATURE" Object contains a signature (using the signing key) of - the PKCS1-padded digest of the entire document, taken from the - beginning of the Initial item, through the newline after the Signature - Item's keyword and its arguments. - - Unless otherwise, the digest algorithm is SHA-1. - - All documents are invalid unless signed with the correct signing key. - - The "Digest" of a document, unless stated otherwise, is its digest *as - signed by this signature scheme*. - -1.4. Voting timeline - - Every consensus document has a "valid-after" (VA) time, a "fresh-until" - (FU) time and a "valid-until" (VU) time. VA MUST precede FU, which MUST - in turn precede VU. Times are chosen so that every consensus will be - "fresh" until the next consensus becomes valid, and "valid" for a while - after. At least 3 consensuses should be valid at any given time. - - The timeline for a given consensus is as follows: - - VA-DistSeconds-VoteSeconds: The authorities exchange votes. - - VA-DistSeconds-VoteSeconds/2: The authorities try to download any - votes they don't have. - - VA-DistSeconds: The authorities calculate the consensus and exchange - signatures. - - VA-DistSeconds/2: The authorities try to download any signatures - they don't have. - - VA: All authorities have a multiply signed consensus. - - VA ... FU: Caches download the consensus. (Note that since caches have - no way of telling what VA and FU are until they have downloaded - the consensus, they assume that the present consensus's VA is - equal to the previous one's FU, and that its FU is one interval after - that.) - - FU: The consensus is no longer the freshest consensus. - - FU ... (the current consensus's VU): Clients download the consensus. - (See note above: clients guess that the next consensus's FU will be - two intervals after the current VA.) - - VU: The consensus is no longer valid. - - VoteSeconds and DistSeconds MUST each be at least 20 seconds; FU-VA and - VU-FU MUST each be at least 5 minutes. - -2. Router operation and formats - - ORs SHOULD generate a new router descriptor and a new extra-info - document whenever any of the following events have occurred: - - - A period of time (18 hrs by default) has passed since the last - time a descriptor was generated. - - - A descriptor field other than bandwidth or uptime has changed. - - - Bandwidth has changed by a factor of 2 from the last time a - descriptor was generated, and at least a given interval of time - (20 mins by default) has passed since then. - - - Its uptime has been reset (by restarting). - - [XXX this list is incomplete; see router_differences_are_cosmetic() - in routerlist.c for others] - - ORs SHOULD NOT publish a new router descriptor or extra-info document - if none of the above events have occurred and not much time has passed - (12 hours by default). - - After generating a descriptor, ORs upload them to every directory - authority they know, by posting them (in order) to the URL - - http:///tor/ - -2.1. Router descriptor format - - Router descriptors consist of the following items. For backward - compatibility, there should be an extra NL at the end of each router - descriptor. - - In lines that take multiple arguments, extra arguments SHOULD be - accepted and ignored. Many of the nonterminals below are defined in - section 2.3. - - "router" nickname address ORPort SOCKSPort DirPort NL - - [At start, exactly once.] - - Indicates the beginning of a router descriptor. "nickname" must be a - valid router nickname as specified in 2.3. "address" must be an IPv4 - address in dotted-quad format. The last three numbers indicate the - TCP ports at which this OR exposes functionality. ORPort is a port at - which this OR accepts TLS connections for the main OR protocol; - SOCKSPort is deprecated and should always be 0; and DirPort is the - port at which this OR accepts directory-related HTTP connections. If - any port is not supported, the value 0 is given instead of a port - number. (At least one of DirPort and ORPort SHOULD be set; - authorities MAY reject any descriptor with both DirPort and ORPort of - 0.) - - "bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed NL - - [Exactly once] - - Estimated bandwidth for this router, in bytes per second. The - "average" bandwidth is the volume per second that the OR is willing to - sustain over long periods; the "burst" bandwidth is the volume that - the OR is willing to sustain in very short intervals. The "observed" - value is an estimate of the capacity this server can handle. The - server remembers the max bandwidth sustained output over any ten - second period in the past day, and another sustained input. The - "observed" value is the lesser of these two numbers. - - "platform" string NL - - [At most once] - - A human-readable string describing the system on which this OR is - running. This MAY include the operating system, and SHOULD include - the name and version of the software implementing the Tor protocol. - - "published" YYYY-MM-DD HH:MM:SS NL - - [Exactly once] - - The time, in GMT, when this descriptor (and its corresponding - extra-info document if any) was generated. - - "fingerprint" fingerprint NL - - [At most once] - - A fingerprint (a HASH_LEN-byte of asn1 encoded public key, encoded in - hex, with a single space after every 4 characters) for this router's - identity key. A descriptor is considered invalid (and MUST be - rejected) if the fingerprint line does not match the public key. - - [We didn't start parsing this line until Tor 0.1.0.6-rc; it should - be marked with "opt" until earlier versions of Tor are obsolete.] - - "hibernating" bool NL - - [At most once] - - If the value is 1, then the Tor server was hibernating when the - descriptor was published, and shouldn't be used to build circuits. - - [We didn't start parsing this line until Tor 0.1.0.6-rc; it should be - marked with "opt" until earlier versions of Tor are obsolete.] - - "uptime" number NL - - [At most once] - - The number of seconds that this OR process has been running. - - "onion-key" NL a public key in PEM format - - [Exactly once] - - This key is used to encrypt EXTEND cells for this OR. The key MUST be - accepted for at least 1 week after any new key is published in a - subsequent descriptor. It MUST be 1024 bits. - - "signing-key" NL a public key in PEM format - - [Exactly once] - - The OR's long-term identity key. It MUST be 1024 bits. - - "accept" exitpattern NL - "reject" exitpattern NL - - [Any number] - - These lines describe an "exit policy": the rules that an OR follows - when deciding whether to allow a new stream to a given address. The - 'exitpattern' syntax is described below. There MUST be at least one - such entry. The rules are considered in order; if no rule matches, - the address will be accepted. For clarity, the last such entry SHOULD - be accept *:* or reject *:*. - - "router-signature" NL Signature NL - - [At end, exactly once] - - The "SIGNATURE" object contains a signature of the PKCS1-padded - hash of the entire router descriptor, taken from the beginning of the - "router" line, through the newline after the "router-signature" line. - The router descriptor is invalid unless the signature is performed - with the router's identity key. - - "contact" info NL - - [At most once] - - Describes a way to contact the server's administrator, preferably - including an email address and a PGP key fingerprint. - - "family" names NL - - [At most once] - - 'Names' is a space-separated list of server nicknames or - hexdigests. If two ORs list one another in their "family" entries, - then OPs should treat them as a single OR for the purpose of path - selection. - - For example, if node A's descriptor contains "family B", and node B's - descriptor contains "family A", then node A and node B should never - be used on the same circuit. - - "read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - [At most once] - "write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - [At most once] - - Declare how much bandwidth the OR has used recently. Usage is divided - into intervals of NSEC seconds. The YYYY-MM-DD HH:MM:SS field - defines the end of the most recent interval. The numbers are the - number of bytes used in the most recent intervals, ordered from - oldest to newest. - - [We didn't start parsing these lines until Tor 0.1.0.6-rc; they should - be marked with "opt" until earlier versions of Tor are obsolete.] - - [See also migration notes in section 2.2.1.] - - "eventdns" bool NL - - [At most once] - - Declare whether this version of Tor is using the newer enhanced - dns logic. Versions of Tor with this field set to false SHOULD NOT - be used for reverse hostname lookups. - - [All versions of Tor before 0.1.2.2-alpha should be assumed to have - this option set to 0 if it is not present. All Tor versions at - 0.1.2.2-alpha or later should be assumed to have this option set to - 1 if it is not present. Until 0.1.2.1-alpha-dev, this option was - not generated, even when the new DNS code was in use. Versions of Tor - before 0.1.2.1-alpha-dev did not parse this option, so it should be - marked "opt". The dnsworker logic has been removed, so this option - should not be used by new server code. However, it can still be - used, and should still be recognized by new code until Tor 0.1.2.x - is obsolete.] - - "caches-extra-info" NL - - [At most once.] - - Present only if this router is a directory cache that provides - extra-info documents. - - [Versions before 0.2.0.1-alpha don't recognize this, and versions - before 0.1.2.5-alpha will reject descriptors containing it unless - it is prefixed with "opt"; it should be so prefixed until these - versions are obsolete.] - - "extra-info-digest" digest NL - - [At most once] - - "Digest" is a hex-encoded digest (using upper-case characters) of the - router's extra-info document, as signed in the router's extra-info - (that is, not including the signature). (If this field is absent, the - router is not uploading a corresponding extra-info document.) - - [Versions before 0.2.0.1-alpha don't recognize this, and versions - before 0.1.2.5-alpha will reject descriptors containing it unless - it is prefixed with "opt"; it should be so prefixed until these - versions are obsolete.] - - "hidden-service-dir" *(SP VersionNum) NL - - [At most once.] - - Present only if this router stores and serves hidden service - descriptors. If any VersionNum(s) are specified, this router - supports those descriptor versions. If none are specified, it - defaults to version 2 descriptors. - - [Versions of Tor before 0.1.2.5-alpha rejected router descriptors - with unrecognized items; the protocols line should be preceded with - an "opt" until these Tors are obsolete.] - - "protocols" SP "Link" SP LINK-VERSION-LIST SP "Circuit" SP - CIRCUIT-VERSION-LIST NL - - [At most once.] - - Both lists are space-separated sequences of numbers, to indicate which - protocols the server supports. As of 30 Mar 2008, specified - protocols are "Link 1 2 Circuit 1". See section 4.1 of tor-spec.txt - for more information about link protocol versions. - - [Versions of Tor before 0.1.2.5-alpha rejected router descriptors - with unrecognized items; the protocols line should be preceded with - an "opt" until these Tors are obsolete.] - - "allow-single-hop-exits" - - [At most one.] - - Present only if the router allows single-hop circuits to make exit - connections. Most Tor servers do not support this: this is - included for specialized controllers designed to support perspective - access and such. - - -2.2. Extra-info documents - - Extra-info documents consist of the following items: - - "extra-info" Nickname Fingerprint NL - [At start, exactly once.] - - Identifies what router this is an extra info descriptor for. - Fingerprint is encoded in hex (using upper-case letters), with - no spaces. - - "published" - - [Exactly once.] - - The time, in GMT, when this document (and its corresponding router - descriptor if any) was generated. It MUST match the published time - in the corresponding router descriptor. - - "read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - [At most once.] - "write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - [At most once.] - - As documented in 2.1 above. See migration notes in section 2.2.1. - - "geoip-start" YYYY-MM-DD HH:MM:SS NL - "geoip-client-origins" CC=N,CC=N,... NL - - Only generated by bridge routers (see blocking.pdf), and only - when they have been configured with a geoip database. - Non-bridges SHOULD NOT generate these fields. Contains a list - of mappings from two-letter country codes (CC) to the number - of clients that have connected to that bridge from that - country (approximate, and rounded up to the nearest multiple of 8 - in order to hamper traffic analysis). A country is included - only if it has at least one address. The time in - "geoip-start" is the time at which we began collecting geoip - statistics. - - "router-signature" NL Signature NL - [At end, exactly once.] - - A document signature as documented in section 1.3, using the - initial item "extra-info" and the final item "router-signature", - signed with the router's identity key. - -2.2.1. Moving history fields to extra-info documents. - - Tools that want to use the read-history and write-history values SHOULD - download extra-info documents as well as router descriptors. Such - tools SHOULD accept history values from both sources; if they appear in - both documents, the values in the extra-info documents are authoritative. - - New versions of Tor no longer generate router descriptors - containing read-history or write-history. Tools should continue to - accept read-history and write-history values in router descriptors - produced by older versions of Tor until all Tor versions earlier - than 0.2.0.x are obsolete. - -2.3. Nonterminals in router descriptors - - nickname ::= between 1 and 19 alphanumeric characters ([A-Za-z0-9]), - case-insensitive. - hexdigest ::= a '$', followed by 40 hexadecimal characters - ([A-Fa-f0-9]). [Represents a server by the digest of its identity - key.] - - exitpattern ::= addrspec ":" portspec - portspec ::= "*" | port | port "-" port - port ::= an integer between 1 and 65535, inclusive. - - [Some implementations incorrectly generate ports with value 0. - Implementations SHOULD accept this, and SHOULD NOT generate it. - Connections to port 0 are never permitted.] - - addrspec ::= "*" | ip4spec | ip6spec - ipv4spec ::= ip4 | ip4 "/" num_ip4_bits | ip4 "/" ip4mask - ip4 ::= an IPv4 address in dotted-quad format - ip4mask ::= an IPv4 mask in dotted-quad format - num_ip4_bits ::= an integer between 0 and 32 - ip6spec ::= ip6 | ip6 "/" num_ip6_bits - ip6 ::= an IPv6 address, surrounded by square brackets. - num_ip6_bits ::= an integer between 0 and 128 - - bool ::= "0" | "1" - -3. Formats produced by directory authorities. - - Every authority has two keys used in this protocol: a signing key, and - an authority identity key. (Authorities also have a router identity - key used in their role as a router and by earlier versions of the - directory protocol.) The identity key is used from time to time to - sign new key certificates using new signing keys; it is very sensitive. - The signing key is used to sign key certificates and status documents. - - There are three kinds of documents generated by directory authorities: - - Key certificates - Status votes - Status consensuses - - Each is discussed below. - -3.1. Key certificates - - Key certificates consist of the following items: - - "dir-key-certificate-version" version NL - - [At start, exactly once.] - - Determines the version of the key certificate. MUST be "3" for - the protocol described in this document. Implementations MUST - reject formats they don't understand. - - "dir-address" IPPort NL - [At most once] - - An IP:Port for this authority's directory port. - - "fingerprint" fingerprint NL - - [Exactly once.] - - Hexadecimal encoding without spaces based on the authority's - identity key. - - "dir-identity-key" NL a public key in PEM format - - [Exactly once.] - - The long-term authority identity key for this authority. This key - SHOULD be at least 2048 bits long; it MUST NOT be shorter than - 1024 bits. - - "dir-key-published" YYYY-MM-DD HH:MM:SS NL - - [Exactly once.] - - The time (in GMT) when this document and corresponding key were - last generated. - - "dir-key-expires" YYYY-MM-DD HH:MM:SS NL - - [Exactly once.] - - A time (in GMT) after which this key is no longer valid. - - "dir-signing-key" NL a key in PEM format - - [Exactly once.] - - The directory server's public signing key. This key MUST be at - least 1024 bits, and MAY be longer. - - "dir-key-crosscert" NL CrossSignature NL - - [At most once.] - - NOTE: Authorities MUST include this field in all newly generated - certificates. A future version of this specification will make - the field required. - - CrossSignature is a signature, made using the certificate's signing - key, of the digest of the PKCS1-padded hash of the certificate's - identity key. For backward compatibility with broken versions of the - parser, we wrap the base64-encoded signature in -----BEGIN ID - SIGNATURE---- and -----END ID SIGNATURE----- tags. Implementations - MUST allow the "ID " portion to be omitted, however. - - When encountering a certificate with a dir-key-crosscert entry, - implementations MUST verify that the signature is a correct signature - of the hash of the identity key using the signing key. - - "dir-key-certification" NL Signature NL - - [At end, exactly once.] - - A document signature as documented in section 1.3, using the - initial item "dir-key-certificate-version" and the final item - "dir-key-certification", signed with the authority identity key. - - Authorities MUST generate a new signing key and corresponding - certificate before the key expires. - -3.2. Vote and consensus status documents - - Votes and consensuses are more strictly formatted then other documents - in this specification, since different authorities must be able to - generate exactly the same consensus given the same set of votes. - - The procedure for deciding when to generate vote and consensus status - documents are described in section XXX below. - - Status documents contain a preamble, an authority section, a list of - router status entries, and one more footers signature, in that order. - - Unlike other formats described above, a SP in these documents must be a - single space character (hex 20). - - Some items appear only in votes, and some items appear only in - consensuses. Unless specified, items occur in both. - - The preamble contains the following items. They MUST occur in the - order given here: - - "network-status-version" SP version NL. - - [At start, exactly once.] - - A document format version. For this specification, the version is - "3". - - "vote-status" SP type NL - - [Exactly once.] - - The status MUST be "vote" or "consensus", depending on the type of - the document. - - "consensus-methods" SP IntegerList NL - - [Exactly once for votes; does not occur in consensuses.] - - A space-separated list of supported methods for generating - consensuses from votes. See section 3.4.1 for details. Method "1" - MUST be included. - - "consensus-method" SP Integer NL - - [Exactly once for consensuses; does not occur in votes.] - - See section 3.4.1 for details. - - (Only included when the vote is generated with consensus-method 2 or - later.) - - "published" SP YYYY-MM-DD SP HH:MM:SS NL - - [Exactly once for votes; does not occur in consensuses.] - - The publication time for this status document (if a vote). - - "valid-after" SP YYYY-MM-DD SP HH:MM:SS NL - - [Exactly once.] - - The start of the Interval for this vote. Before this time, the - consensus document produced from this vote should not be used. - See 1.4 for voting timeline information. - - "fresh-until" SP YYYY-MM-DD SP HH:MM:SS NL - - [Exactly once.] - - The time at which the next consensus should be produced; before this - time, there is no point in downloading another consensus, since there - won't be a new one. See 1.4 for voting timeline information. - - "valid-until" SP YYYY-MM-DD SP HH:MM:SS NL - - [Exactly once.] - - The end of the Interval for this vote. After this time, the - consensus produced by this vote should not be used. See 1.4 for - voting timeline information. - - "voting-delay" SP VoteSeconds SP DistSeconds NL - - [Exactly once.] - - VoteSeconds is the number of seconds that we will allow to collect - votes from all authorities; DistSeconds is the number of seconds - we'll allow to collect signatures from all authorities. See 1.4 for - voting timeline information. - - "client-versions" SP VersionList NL - - [At most once.] - - A comma-separated list of recommended client versions, in - ascending order. If absent, no opinion is held about client - versions. - - "server-versions" SP VersionList NL - - [At most once.] - - A comma-separated list of recommended server versions, in - ascending order. If absent, no opinion is held about server - versions. - - "known-flags" SP FlagList NL - - [Exactly once.] - - A space-separated list of all of the flags that this document - might contain. A flag is "known" either because the authority - knows about them and might set them (if in a vote), or because - enough votes were counted for the consensus for an authoritative - opinion to have been formed about their status. - - - The authority section of a vote contains the following items, followed - in turn by the authority's current key certificate: - - "dir-source" SP nickname SP identity SP address SP IP SP dirport SP - orport NL - - [Exactly once, at start] - - Describes this authority. The nickname is a convenient identifier - for the authority. The identity is an uppercase hex fingerprint of - the authority's current (v3 authority) identity key. The address is - the server's hostname. The IP is the server's current IP address, - and dirport is its current directory port. XXXXorport - - "contact" SP string NL - - [At most once.] - - An arbitrary string describing how to contact the directory - server's administrator. Administrators should include at least an - email address and a PGP fingerprint. - - "legacy-key" SP FINGERPRINT NL - - [At most once] - - Lists a fingerprint for an obsolete _identity_ key still used - by this authority to keep older clients working. This option - is used to keep key around for a little while in case the - authorities need to migrate many identity keys at once. - (Generally, this would only happen because of a security - vulnerability that affected multiple authorities, like the - Debian OpenSSL RNG bug of May 2008.) - - The authority section of a consensus contains groups the following items, - in the order given, with one group for each authority that contributed to - the consensus, with groups sorted by authority identity digest: - - "dir-source" SP nickname SP identity SP address SP IP SP dirport SP - orport NL - - [Exactly once, at start] - - As in the authority section of a vote. - - "contact" SP string NL - - [At most once.] - - As in the authority section of a vote. - - "vote-digest" SP digest NL - - [Exactly once.] - - A digest of the vote from the authority that contributed to this - consensus, as signed (that is, not including the signature). - (Hex, upper-case.) - - Each router status entry contains the following items. Router status - entries are sorted in ascending order by identity digest. - - "r" SP nickname SP identity SP digest SP publication SP IP SP ORPort - SP DirPort NL - - [At start, exactly once.] - - "Nickname" is the OR's nickname. "Identity" is a hash of its - identity key, encoded in base64, with trailing equals sign(s) - removed. "Digest" is a hash of its most recent descriptor as - signed (that is, not including the signature), encoded in base64. - "Publication" is the - publication time of its most recent descriptor, in the form - YYYY-MM-DD HH:MM:SS, in GMT. "IP" is its current IP address; - ORPort is its current OR port, "DirPort" is it's current directory - port, or "0" for "none". - - "s" SP Flags NL - - [At most once.] - - A series of space-separated status flags, in alphabetical order. - Currently documented flags are: - - "Authority" if the router is a directory authority. - "BadExit" if the router is believed to be useless as an exit node - (because its ISP censors it, because it is behind a restrictive - proxy, or for some similar reason). - "BadDirectory" if the router is believed to be useless as a - directory cache (because its directory port isn't working, - its bandwidth is always throttled, or for some similar - reason). - "Exit" if the router is more useful for building - general-purpose exit circuits than for relay circuits. The - path building algorithm uses this flag; see path-spec.txt. - "Fast" if the router is suitable for high-bandwidth circuits. - "Guard" if the router is suitable for use as an entry guard. - "HSDir" if the router is considered a v2 hidden service directory. - "Named" if the router's identity-nickname mapping is canonical, - and this authority binds names. - "Stable" if the router is suitable for long-lived circuits. - "Running" if the router is currently usable. - "Unnamed" if another router has bound the name used by this - router, and this authority binds names. - "Valid" if the router has been 'validated'. - "V2Dir" if the router implements the v2 directory protocol. - "V3Dir" if the router implements this protocol. - - "v" SP version NL - - [At most once.] - - The version of the Tor protocol that this server is running. If - the value begins with "Tor" SP, the rest of the string is a Tor - version number, and the protocol is "The Tor protocol as supported - by the given version of Tor." Otherwise, if the value begins with - some other string, Tor has upgraded to a more sophisticated - protocol versioning system, and the protocol is "a version of the - Tor protocol more recent than any we recognize." - - Directory authorities SHOULD omit version strings they receive from - descriptors if they would cause "v" lines to be over 128 characters - long. - - "w" SP "Bandwidth=" INT NL - - [At most once.] - - An estimate of the bandwidth of this server, in an arbitrary - unit (currently kilobytes per second). Used to weight router - selection. Other weighting keywords may be added later. - Clients MUST ignore keywords they do not recognize. - - "p" SP ("accept" / "reject") SP PortList NL - - [At most once.] - - PortList = PortOrRange - PortList = PortList "," PortOrRange - PortOrRange = INT "-" INT / INT - - A list of those ports that this router supports (if 'accept') - or does not support (if 'reject') for exit to "most - addresses". - - The signature section contains the following item, which appears - Exactly Once for a vote, and At Least Once for a consensus. - - "directory-signature" SP identity SP signing-key-digest NL Signature - - This is a signature of the status document, with the initial item - "network-status-version", and the signature item - "directory-signature", using the signing key. (In this case, we take - the hash through the _space_ after directory-signature, not the - newline: this ensures that all authorities sign the same thing.) - "identity" is the hex-encoded digest of the authority identity key of - the signing authority, and "signing-key-digest" is the hex-encoded - digest of the current authority signing key of the signing authority. - -3.3. Deciding how to vote. - - (This section describes how directory authorities choose which status - flags to apply to routers, as of Tor 0.2.0.0-alpha-dev. Later directory - authorities MAY do things differently, so long as clients keep working - well. Clients MUST NOT depend on the exact behaviors in this section.) - - In the below definitions, a router is considered "active" if it is - running, valid, and not hibernating. - - "Valid" -- a router is 'Valid' if it is running a version of Tor not - known to be broken, and the directory authority has not blacklisted - it as suspicious. - - "Named" -- Directory authority administrators may decide to support name - binding. If they do, then they must maintain a file of - nickname-to-identity-key mappings, and try to keep this file consistent - with other directory authorities. If they don't, they act as clients, and - report bindings made by other directory authorities (name X is bound to - identity Y if at least one binding directory lists it, and no directory - binds X to some other Y'.) A router is called 'Named' if the router - believes the given name should be bound to the given key. - - Two strategies exist on the current network for deciding on - values for the Named flag. In the original version, server - operators were asked to send nickname-identity pairs to a - mailing list of Naming directory authorities operators. The - operators were then supposed to add the pairs to their - mapping files; in practice, they didn't get to this often. - - Newer Naming authorities run a script that registers routers - in their mapping files once the routers have been online at - least two weeks, no other router has that nickname, and no - other router has wanted the nickname for a month. If a router - has not been online for six months, the router is removed. - - "Unnamed" -- Directory authorities that support naming should vote for a - router to be 'Unnamed' if its given nickname is mapped to a different - identity. - - "Running" -- A router is 'Running' if the authority managed to connect to - it successfully within the last 30 minutes. - - "Stable" -- A router is 'Stable' if it is active, and either its Weighted - MTBF is at least the median for known active routers or its Weighted MTBF - corresponds to at least 7 days. Routers are never called Stable if they are - running a version of Tor known to drop circuits stupidly. (0.1.1.10-alpha - through 0.1.1.16-rc are stupid this way.) - - To calculate weighted MTBF, compute the weighted mean of the lengths - of all intervals when the router was observed to be up, weighting - intervals by $\alpha^n$, where $n$ is the amount of time that has - passed since the interval ended, and $\alpha$ is chosen so that - measurements over approximately one month old no longer influence the - weighted MTBF much. - - [XXXX what happens when we have less than 4 days of MTBF info.] - - "Exit" -- A router is called an 'Exit' iff it allows exits to at - least two of the ports 80, 443, and 6667 and allows exits to at - least one /8 address space. - - "Fast" -- A router is 'Fast' if it is active, and its bandwidth is - either in the top 7/8ths for known active routers or at least 100KB/s. - - "Guard" -- A router is a possible 'Guard' if its Weighted Fractional - Uptime is at least the median for "familiar" active routers, and if - its bandwidth is at least median or at least 250KB/s. - If the total bandwidth of active non-BadExit Exit servers is less - than one third of the total bandwidth of all active servers, no Exit is - listed as a Guard. - - To calculate weighted fractional uptime, compute the fraction - of time that the router is up in any given day, weighting so that - downtime and uptime in the past counts less. - - A node is 'familiar' if 1/8 of all active nodes have appeared more - recently than it, OR it has been around for a few weeks. - - "Authority" -- A router is called an 'Authority' if the authority - generating the network-status document believes it is an authority. - - "V2Dir" -- A router supports the v2 directory protocol if it has an open - directory port, and it is running a version of the directory protocol that - supports the functionality clients need. (Currently, this is - 0.1.1.9-alpha or later.) - - "V3Dir" -- A router supports the v3 directory protocol if it has an open - directory port, and it is running a version of the directory protocol that - supports the functionality clients need. (Currently, this is - 0.2.0.?????-alpha or later.) - - "HSDir" -- A router is a v2 hidden service directory if it stores and - serves v2 hidden service descriptors and the authority managed to connect - to it successfully within the last 24 hours. - - Directory server administrators may label some servers or IPs as - blacklisted, and elect not to include them in their network-status lists. - - Authorities SHOULD 'disable' any servers in excess of 3 on any single IP. - When there are more than 3 to choose from, authorities should first prefer - authorities to non-authorities, then prefer Running to non-Running, and - then prefer high-bandwidth to low-bandwidth. To 'disable' a server, the - authority *should* advertise it without the Running or Valid flag. - - Thus, the network-status vote includes all non-blacklisted, - non-expired, non-superseded descriptors. - - The bandwidth in a "w" line should be taken as the best estimate - of the router's actual capacity that the authority has. For now, - this should be the lesser of the observed bandwidth and bandwidth - rate limit from the router descriptor. It is given in kilobytes - per second, and capped at some arbitrary value (currently 10 MB/s). - - The ports listed in a "p" line should be taken as those ports for - which the router's exit policy permits 'most' addresses, ignoring any - accept not for all addresses, ignoring all rejects for private - netblocks. "Most" addresses are permitted if no more than 2^25 - IPv4 addresses (two /8 networks) were blocked. The list is encoded - as described in 3.4.2. - -3.4. Computing a consensus from a set of votes - - Given a set of votes, authorities compute the contents of the consensus - document as follows: - - The "valid-after", "valid-until", and "fresh-until" times are taken as - the median of the respective values from all the votes. - - The times in the "voting-delay" line are taken as the median of the - VoteSeconds and DistSeconds times in the votes. - - Known-flags is the union of all flags known by any voter. - - "client-versions" and "server-versions" are sorted in ascending - order; A version is recommended in the consensus if it is recommended - by more than half of the voting authorities that included a - client-versions or server-versions lines in their votes. - - The authority item groups (dir-source, contact, fingerprint, - vote-digest) are taken from the votes of the voting - authorities. These groups are sorted by the digests of the - authorities identity keys, in ascending order. If the consensus - method is 3 or later, a dir-source line must be included for - every vote with legacy-key entry, using the legacy-key's - fingerprint, the voter's ordinary nickname with the string - "-legacy" appended, and all other fields as from the original - vote's dir-source line. - - A router status entry: - * is included in the result if some router status entry with the same - identity is included by more than half of the authorities (total - authorities, not just those whose votes we have). - - * For any given identity, we include at most one router status entry. - - * A router entry has a flag set if that is included by more than half - of the authorities who care about that flag. - - * Two router entries are "the same" if they have the same - tuple. - We choose the tuple for a given router as whichever tuple appears - for that router in the most votes. We break ties first in favor of - the more recently published, then in favor of smaller server - descriptor digest. - - * The Named flag appears if it is included for this routerstatus by - _any_ authority, and if all authorities that list it list the same - nickname. However, if consensus-method 2 or later is in use, and - any authority calls this identity/nickname pair Unnamed, then - this routerstatus does not get the Named flag. - - * If consensus-method 2 or later is in use, the Unnamed flag is - set for a routerstatus if any authorities have voted for a different - identities to be Named with that nickname, or if any authority - lists that nickname/ID pair as Unnamed. - - (With consensus-method 1, Unnamed is set like any other flag.) - - * The version is given as whichever version is listed by the most - voters, with ties decided in favor of more recent versions. - - * If consensus-method 4 or later is in use, then routers that - do not have the Running flag are not listed at all. - - * If consensus-method 5 or later is in use, then the "w" line - is generated using a low-median of the bandwidth values from - the votes that included "w" lines for this router. - - * If consensus-method 5 or later is in use, then the "p" line - is taken from the votes that have the same policy summary - for the descriptor we are listing. (They should all be the - same. If they are not, we pick the most commonly listed - one, breaking ties in favor of the lexicographically larger - vote.) The port list is encoded as specified in 3.4.2. - - The signatures at the end of a consensus document are sorted in - ascending order by identity digest. - - All ties in computing medians are broken in favor of the smaller or - earlier item. - -3.4.1. Forward compatibility - - Future versions of Tor will need to include new information in the - consensus documents, but it is important that all authorities (or at least - half) generate and sign the same signed consensus. - - To achieve this, authorities list in their votes their supported methods - for generating consensuses from votes. Later methods will be assigned - higher numbers. Currently recognized methods: - "1" -- The first implemented version. - "2" -- Added support for the Unnamed flag. - "3" -- Added legacy ID key support to aid in authority ID key rollovers - "4" -- No longer list routers that are not running in the consensus - "5" -- adds support for "w" and "p" lines. - - Before generating a consensus, an authority must decide which consensus - method to use. To do this, it looks for the highest version number - supported by more than 2/3 of the authorities voting. If it supports this - method, then it uses it. Otherwise, it falls back to method 1. - - (The consensuses generated by new methods must be parsable by - implementations that only understand the old methods, and must not cause - those implementations to compromise their anonymity. This is a means for - making changes in the contents of consensus; not for making - backward-incompatible changes in their format.) - -3.4.2. Encoding port lists - - Whether the summary shows the list of accepted ports or the list of - rejected ports depends on which list is shorter (has a shorter string - representation). In case of ties we choose the list of accepted - ports. As an exception to this rule an allow-all policy is - represented as "accept 1-65535" instead of "reject " and a reject-all - policy is similarly given as "reject 1-65535". - - Summary items are compressed, that is instead of "80-88,89-100" there - only is a single item of "80-100", similarly instead of "20,21" a - summary will say "20-21". - - Port lists are sorted in ascending order. - - The maximum allowed length of a policy summary (including the "accept " - or "reject ") is 1000 characters. If a summary exceeds that length we - use an accept-style summary and list as much of the port list as is - possible within these 1000 bytes. [XXXX be more specific.] - -3.5. Detached signatures - - Assuming full connectivity, every authority should compute and sign the - same consensus directory in each period. Therefore, it isn't necessary to - download the consensus computed by each authority; instead, the - authorities only push/fetch each others' signatures. A "detached - signature" document contains items as follows: - - "consensus-digest" SP Digest NL - - [At start, at most once.] - - The digest of the consensus being signed. - - "valid-after" SP YYYY-MM-DD SP HH:MM:SS NL - "fresh-until" SP YYYY-MM-DD SP HH:MM:SS NL - "valid-until" SP YYYY-MM-DD SP HH:MM:SS NL - - [As in the consensus] - - "directory-signature" - - [As in the consensus; the signature object is the same as in the - consensus document.] - - -4. Directory server operation - - All directory authorities and directory caches ("directory servers") - implement this section, except as noted. - -4.1. Accepting uploads (authorities only) - - When a router posts a signed descriptor to a directory authority, the - authority first checks whether it is well-formed and correctly - self-signed. If it is, the authority next verifies that the nickname - in question is not already assigned to a router with a different - public key. - Finally, the authority MAY check that the router is not blacklisted - because of its key, IP, or another reason. - - If the descriptor passes these tests, and the authority does not already - have a descriptor for a router with this public key, it accepts the - descriptor and remembers it. - - If the authority _does_ have a descriptor with the same public key, the - newly uploaded descriptor is remembered if its publication time is more - recent than the most recent old descriptor for that router, and either: - - There are non-cosmetic differences between the old descriptor and the - new one. - - Enough time has passed between the descriptors' publication times. - (Currently, 12 hours.) - - Differences between router descriptors are "non-cosmetic" if they would be - sufficient to force an upload as described in section 2 above. - - Note that the "cosmetic difference" test only applies to uploaded - descriptors, not to descriptors that the authority downloads from other - authorities. - - When a router posts a signed extra-info document to a directory authority, - the authority again checks it for well-formedness and correct signature, - and checks that its matches the extra-info-digest in some router - descriptor that it believes is currently useful. If so, it accepts it and - stores it and serves it as requested. If not, it drops it. - -4.2. Voting (authorities only) - - Authorities divide time into Intervals. Authority administrators SHOULD - try to all pick the same interval length, and SHOULD pick intervals that - are commonly used divisions of time (e.g., 5 minutes, 15 minutes, 30 - minutes, 60 minutes, 90 minutes). Voting intervals SHOULD be chosen to - divide evenly into a 24-hour day. - - Authorities SHOULD act according to interval and delays in the - latest consensus. Lacking a latest consensus, they SHOULD default to a - 30-minute Interval, a 5 minute VotingDelay, and a 5 minute DistDelay. - - Authorities MUST take pains to ensure that their clocks remain accurate - within a few seconds. (Running NTP is usually sufficient.) - - The first voting period of each day begins at 00:00 (midnight) GMT. If - the last period of the day would be truncated by one-half or more, it is - merged with the second-to-last period. - - An authority SHOULD publish its vote immediately at the start of each voting - period (minus VoteSeconds+DistSeconds). It does this by making it - available at - http:///tor/status-vote/next/authority.z - and sending it in an HTTP POST request to each other authority at the URL - http:///tor/post/vote - - If, at the start of the voting period, minus DistSeconds, an authority - does not have a current statement from another authority, the first - authority downloads the other's statement. - - Once an authority has a vote from another authority, it makes it available - at - http:///tor/status-vote/next/.z - where is the fingerprint of the other authority's identity key. - And at - http:///tor/status-vote/next/d/.z - where is the digest of the vote document. - - The consensus status, along with as many signatures as the server - currently knows, should be available at - http:///tor/status-vote/next/consensus.z - All of the detached signatures it knows for consensus status should be - available at: - http:///tor/status-vote/next/consensus-signatures.z - - Once there are enough signatures, or once the voting period starts, - these documents are available at - http:///tor/status-vote/current/consensus.z - and - http:///tor/status-vote/current/consensus-signatures.z - [XXX current/consensus-signatures is not currently implemented, as it - is not used in the voting protocol.] - - The other vote documents are analogously made available under - http:///tor/status-vote/current/authority.z - http:///tor/status-vote/current/.z - http:///tor/status-vote/current/d/.z - once the consensus is complete. - - Once an authority has computed and signed a consensus network status, it - should send its detached signature to each other authority in an HTTP POST - request to the URL: - http:///tor/post/consensus-signature - - [XXX Note why we support push-and-then-pull.] - - [XXX possible future features include support for downloading old - consensuses.] - -4.3. Downloading consensus status documents (caches only) - - All directory servers (authorities and caches) try to keep a recent - network-status consensus document to serve to clients. A cache ALWAYS - downloads a network-status consensus if any of the following are true: - - The cache has no consensus document. - - The cache's consensus document is no longer valid. - Otherwise, the cache downloads a new consensus document at a randomly - chosen time in the first half-interval after its current consensus - stops being fresh. (This time is chosen at random to avoid swarming - the authorities at the start of each period. The interval size is - inferred from the difference between the valid-after time and the - fresh-until time on the consensus.) - - [For example, if a cache has a consensus that became valid at 1:00, - and is fresh until 2:00, that cache will fetch a new consensus at - a random time between 2:00 and 2:30.] - -4.4. Downloading and storing router descriptors (authorities and caches) - - Periodically (currently, every 10 seconds), directory servers check - whether there are any specific descriptors that they do not have and that - they are not currently trying to download. Caches identify these - descriptors by hash in the recent network-status consensus documents; - authorities identify them by hash in vote (if publication date is more - recent than the descriptor we currently have). - - [XXXX need a way to fetch descriptors ahead of the vote? v2 status docs can - do that for now.] - - If so, the directory server launches requests to the authorities for these - descriptors, such that each authority is only asked for descriptors listed - in its most recent vote (if the requester is an authority) or in the - consensus (if the requester is a cache). If we're an authority, and more - than one authority lists the descriptor, we choose which to ask at random. - - If one of these downloads fails, we do not try to download that descriptor - from the authority that failed to serve it again unless we receive a newer - network-status (consensus or vote) from that authority that lists the same - descriptor. - - Directory servers must potentially cache multiple descriptors for each - router. Servers must not discard any descriptor listed by any recent - consensus. If there is enough space to store additional descriptors, - servers SHOULD try to hold those which clients are likely to download the - most. (Currently, this is judged based on the interval for which each - descriptor seemed newest.) -[XXXX define recent] - - Authorities SHOULD NOT download descriptors for routers that they would - immediately reject for reasons listed in 3.1. - -4.5. Downloading and storing extra-info documents - - All authorities, and any cache that chooses to cache extra-info documents, - and any client that uses extra-info documents, should implement this - section. - - Note that generally, clients don't need extra-info documents. - - Periodically, the Tor instance checks whether it is missing any extra-info - documents: in other words, if it has any router descriptors with an - extra-info-digest field that does not match any of the extra-info - documents currently held. If so, it downloads whatever extra-info - documents are missing. Caches download from authorities; non-caches try - to download from caches. We follow the same splitting and back-off rules - as in 4.4 (if a cache) or 5.3 (if a client). - -4.6. General-use HTTP URLs - - "Fingerprints" in these URLs are base-16-encoded SHA1 hashes. - - The most recent v3 consensus should be available at: - http:///tor/status-vote/current/consensus.z - - Starting with Tor version 0.2.1.1-alpha is also available at: - http:///tor/status-vote/current/consensus/++.z - - Where F1, F2, etc. are authority identity fingerprints the client trusts. - Servers will only return a consensus if more than half of the requested - authorities have signed the document, otherwise a 404 error will be sent - back. The fingerprints can be shortened to a length of any multiple of - two, using only the leftmost part of the encoded fingerprint. Tor uses - 3 bytes (6 hex characters) of the fingerprint. - - Clients SHOULD sort the fingerprints in ascending order. Server MUST - accept any order. - - Clients SHOULD use this format when requesting consensus documents from - directory authority servers and from caches running a version of Tor - that is known to support this URL format. - - A concatenated set of all the current key certificates should be available - at: - http:///tor/keys/all.z - - The key certificate for this server (if it is an authority) should be - available at: - http:///tor/keys/authority.z - - The key certificate for an authority whose authority identity fingerprint - is should be available at: - http:///tor/keys/fp/.z - - The key certificate whose signing key fingerprint is should be - available at: - http:///tor/keys/sk/.z - - The key certificate whose identity key fingerprint is and whose signing - key fingerprint is should be available at: - - http:///tor/keys/fp-sk/-.z - - (As usual, clients may request multiple certificates using: - http:///tor/keys/fp-sk/-+-.z ) - [The above fp-sk format was not supported before Tor 0.2.1.9-alpha.] - - The most recent descriptor for a server whose identity key has a - fingerprint of should be available at: - http:///tor/server/fp/.z - - The most recent descriptors for servers with identity fingerprints - ,, should be available at: - http:///tor/server/fp/++.z - - (NOTE: Implementations SHOULD NOT download descriptors by identity key - fingerprint. This allows a corrupted server (in collusion with a cache) to - provide a unique descriptor to a client, and thereby partition that client - from the rest of the network.) - - The server descriptor with (descriptor) digest (in hex) should be - available at: - http:///tor/server/d/.z - - The most recent descriptors with digests ,, should be - available at: - http:///tor/server/d/++.z - - The most recent descriptor for this server should be at: - http:///tor/server/authority.z - [Nothing in the Tor protocol uses this resource yet, but it is useful - for debugging purposes. Also, the official Tor implementations - (starting at 0.1.1.x) use this resource to test whether a server's - own DirPort is reachable.] - - A concatenated set of the most recent descriptors for all known servers - should be available at: - http:///tor/server/all.z - - Extra-info documents are available at the URLS - http:///tor/extra/d/... - http:///tor/extra/fp/... - http:///tor/extra/all[.z] - http:///tor/extra/authority[.z] - (As for /tor/server/ URLs: supports fetching extra-info - documents by their digest, by the fingerprint of their servers, - or all at once. When serving by fingerprint, we serve the - extra-info that corresponds to the descriptor we would serve by - that fingerprint. Only directory authorities of version - 0.2.0.1-alpha or later are guaranteed to support the first - three classes of URLs. Caches may support them, and MUST - support them if they have advertised "caches-extra-info".) - - For debugging, directories SHOULD expose non-compressed objects at URLs like - the above, but without the final ".z". - Clients MUST handle compressed concatenated information in two forms: - - A concatenated list of zlib-compressed objects. - - A zlib-compressed concatenated list of objects. - Directory servers MAY generate either format: the former requires less - CPU, but the latter requires less bandwidth. - - Clients SHOULD use upper case letters (A-F) when base16-encoding - fingerprints. Servers MUST accept both upper and lower case fingerprints - in requests. - -5. Client operation: downloading information - - Every Tor that is not a directory server (that is, those that do - not have a DirPort set) implements this section. - -5.1. Downloading network-status documents - - Each client maintains a list of directory authorities. Insofar as - possible, clients SHOULD all use the same list. - - Clients try to have a live consensus network-status document at all times. - A network-status document is "live" if the time in its valid-until field - has not passed. - - If a client is missing a live network-status document, it tries to fetch - it from a directory cache (or from an authority if it knows no caches). - On failure, the client waits briefly, then tries that network-status - document again from another cache. The client does not build circuits - until it has a live network-status consensus document, and it has - descriptors for more than 1/4 of the routers that it believes are running. - - (Note: clients can and should pick caches based on the network-status - information they have: once they have first fetched network-status info - from an authority, they should not need to go to the authority directly - again.) - - To avoid swarming the caches whenever a consensus expires, the - clients download new consensuses at a randomly chosen time after the - caches are expected to have a fresh consensus, but before their - consensus will expire. (This time is chosen uniformly at random from - the interval between the time 3/4 into the first interval after the - consensus is no longer fresh, and 7/8 of the time remaining after - that before the consensus is invalid.) - - [For example, if a cache has a consensus that became valid at 1:00, - and is fresh until 2:00, and expires at 4:00, that cache will fetch - a new consensus at a random time between 2:45 and 3:50, since 3/4 - of the one-hour interval is 45 minutes, and 7/8 of the remaining 75 - minutes is 65 minutes.] - -5.2. Downloading and storing router descriptors - - Clients try to have the best descriptor for each router. A descriptor is - "best" if: - * It is listed in the consensus network-status document. - - Periodically (currently every 10 seconds) clients check whether there are - any "downloadable" descriptors. A descriptor is downloadable if: - - It is the "best" descriptor for some router. - - The descriptor was published at least 10 minutes in the past. - (This prevents clients from trying to fetch descriptors that the - mirrors have probably not yet retrieved and cached.) - - The client does not currently have it. - - The client is not currently trying to download it. - - The client would not discard it immediately upon receiving it. - - The client thinks it is running and valid (see 6.1 below). - - If at least 16 known routers have downloadable descriptors, or if - enough time (currently 10 minutes) has passed since the last time the - client tried to download descriptors, it launches requests for all - downloadable descriptors, as described in 5.3 below. - - When a descriptor download fails, the client notes it, and does not - consider the descriptor downloadable again until a certain amount of time - has passed. (Currently 0 seconds for the first failure, 60 seconds for the - second, 5 minutes for the third, 10 minutes for the fourth, and 1 day - thereafter.) Periodically (currently once an hour) clients reset the - failure count. - - Clients retain the most recent descriptor they have downloaded for each - router so long as it is not too old (currently, 48 hours), OR so long as - no better descriptor has been downloaded for the same router. - - [Versions of Tor before 0.1.2.3-alpha would discard descriptors simply for - being published too far in the past.] [The code seems to discard - descriptors in all cases after they're 5 days old. True? -RD] - -5.3. Managing downloads - - When a client has no consensus network-status document, it downloads it - from a randomly chosen authority. In all other cases, the client - downloads from caches randomly chosen from among those believed to be V2 - directory servers. (This information comes from the network-status - documents; see 6 below.) - - When downloading multiple router descriptors, the client chooses multiple - mirrors so that: - - At least 3 different mirrors are used, except when this would result - in more than one request for under 4 descriptors. - - No more than 128 descriptors are requested from a single mirror. - - Otherwise, as few mirrors as possible are used. - After choosing mirrors, the client divides the descriptors among them - randomly. - - After receiving any response client MUST discard any network-status - documents and descriptors that it did not request. - -6. Using directory information - - Everyone besides directory authorities uses the approaches in this section - to decide which servers to use and what their keys are likely to be. - (Directory authorities just believe their own opinions, as in 3.1 above.) - -6.1. Choosing routers for circuits. - - Circuits SHOULD NOT be built until the client has enough directory - information: a live consensus network status [XXXX fallback?] and - descriptors for at least 1/4 of the servers believed to be running. - - A server is "listed" if it is included by the consensus network-status - document. Clients SHOULD NOT use unlisted servers. - - These flags are used as follows: - - - Clients SHOULD NOT use non-'Valid' or non-'Running' routers unless - requested to do so. - - - Clients SHOULD NOT use non-'Fast' routers for any purpose other than - very-low-bandwidth circuits (such as introduction circuits). - - - Clients SHOULD NOT use non-'Stable' routers for circuits that are - likely to need to be open for a very long time (such as those used for - IRC or SSH connections). - - - Clients SHOULD NOT choose non-'Guard' nodes when picking entry guard - nodes. - - - Clients SHOULD NOT download directory information from non-'V2Dir' - caches. - - See the "path-spec.txt" document for more details. - -6.2. Managing naming - - In order to provide human-memorable names for individual server - identities, some directory servers bind names to IDs. Clients handle - names in two ways: - - When a client encounters a name it has not mapped before: - - If the consensus lists any router with that name as "Named", or if - consensus-method 2 or later is in use and the consensus lists any - router with that name as having the "Unnamed" flag, then the name is - bound. (It's bound to the ID listed in the entry with the Named, - or to an unknown ID if no name is found.) - - When the user refers to a bound name, the implementation SHOULD provide - only the router with ID bound to that name, and no other router, even - if the router with the right ID can't be found. - - When a user tries to refer to a non-bound name, the implementation SHOULD - warn the user. After warning the user, the implementation MAY use any - router that advertises the name. - - Not every router needs a nickname. When a router doesn't configure a - nickname, it publishes with the default nickname "Unnamed". Authorities - SHOULD NOT ever mark a router with this nickname as Named; client software - SHOULD NOT ever use a router in response to a user request for a router - called "Unnamed". - -6.3. Software versions - - An implementation of Tor SHOULD warn when it has fetched a consensus - network-status, and it is running a software version not listed. - -6.4. Warning about a router's status. - - If a router tries to publish its descriptor to a Naming authority - that has its nickname mapped to another key, the router SHOULD - warn the operator that it is either using the wrong key or is using - an already claimed nickname. - - If a router has fetched a consensus document,, and the - authorities do not publish a binding for the router's nickname, the - router MAY remind the operator that the chosen nickname is not - bound to this key at the authorities, and suggest contacting the - authority operators. - - ... - -6.5. Router protocol versions - - A client should believe that a router supports a given feature if that - feature is supported by the router or protocol versions in more than half - of the live networkstatuses' "v" entries for that router. In other words, - if the "v" entries for some router are: - v Tor 0.0.8pre1 (from authority 1) - v Tor 0.1.2.11 (from authority 2) - v FutureProtocolDescription 99 (from authority 3) - then the client should believe that the router supports any feature - supported by 0.1.2.11. - - This is currently equivalent to believing the median declared version for - a router in all live networkstatuses. - -7. Standards compliance - - All clients and servers MUST support HTTP 1.0. Clients and servers MAY - support later versions of HTTP as well. - -7.1. HTTP headers - - Servers MAY set the Content-Length: header. Servers SHOULD set - Content-Encoding to "deflate" or "identity". - - Servers MAY include an X-Your-Address-Is: header, whose value is the - apparent IP address of the client connecting to them (as a dotted quad). - For directory connections tunneled over a BEGIN_DIR stream, servers SHOULD - report the IP from which the circuit carrying the BEGIN_DIR stream reached - them. [Servers before version 0.1.2.5-alpha reported 127.0.0.1 for all - BEGIN_DIR-tunneled connections.] - - Servers SHOULD disable caching of multiple network statuses or multiple - router descriptors. Servers MAY enable caching of single descriptors, - single network statuses, the list of all router descriptors, a v1 - directory, or a v1 running routers document. XXX mention times. - -7.2. HTTP status codes - - Tor delivers the following status codes. Some were chosen without much - thought; other code SHOULD NOT rely on specific status codes yet. - - 200 -- the operation completed successfully - -- the user requested statuses or serverdescs, and none of the ones we - requested were found (0.2.0.4-alpha and earlier). - - 304 -- the client specified an if-modified-since time, and none of the - requested resources have changed since that time. - - 400 -- the request is malformed, or - -- the URL is for a malformed variation of one of the URLs we support, - or - -- the client tried to post to a non-authority, or - -- the authority rejected a malformed posted document, or - - 404 -- the requested document was not found. - -- the user requested statuses or serverdescs, and none of the ones - requested were found (0.2.0.5-alpha and later). - - 503 -- we are declining the request in order to save bandwidth - -- user requested some items that we ordinarily generate or store, - but we do not have any available. - -9. Backward compatibility and migration plans - - Until Tor versions before 0.1.1.x are completely obsolete, directory - authorities should generate, and mirrors should download and cache, v1 - directories and running-routers lists, and allow old clients to download - them. These documents and the rules for retrieving, serving, and caching - them are described in dir-spec-v1.txt. - - Until Tor versions before 0.2.0.x are completely obsolete, directory - authorities should generate, mirrors should download and cache, v2 - network-status documents, and allow old clients to download them. - Additionally, all directory servers and caches should download, store, and - serve any router descriptor that is required because of v2 network-status - documents. These documents and the rules for retrieving, serving, and - caching them are described in dir-spec-v1.txt. - -A. Consensus-negotiation timeline. - - - Period begins: this is the Published time. - Everybody sends votes - Reconciliation: everybody tries to fetch missing votes. - consensus may exist at this point. - End of voting period: - everyone swaps signatures. - Now it's okay for caches to download - Now it's okay for clients to download. - - Valid-after/valid-until switchover - diff --git a/doc/spec/path-spec.txt b/doc/spec/path-spec.txt deleted file mode 100644 index dceb21dad..000000000 --- a/doc/spec/path-spec.txt +++ /dev/null @@ -1,423 +0,0 @@ -$Id$ - - Tor Path Specification - - Roger Dingledine - Nick Mathewson - -Note: This is an attempt to specify Tor as currently implemented. Future -versions of Tor will implement improved algorithms. - -This document tries to cover how Tor chooses to build circuits and assign -streams to circuits. Other implementations MAY take other approaches, but -implementors should be aware of the anonymity and load-balancing implications -of their choices. - - THIS SPEC ISN'T DONE YET. - -1. General operation - - Tor begins building circuits as soon as it has enough directory - information to do so (see section 5 of dir-spec.txt). Some circuits are - built preemptively because we expect to need them later (for user - traffic), and some are built because of immediate need (for user traffic - that no current circuit can handle, for testing the network or our - reachability, and so on). - - When a client application creates a new stream (by opening a SOCKS - connection or launching a resolve request), we attach it to an appropriate - open circuit if one exists, or wait if an appropriate circuit is - in-progress. We launch a new circuit only - if no current circuit can handle the request. We rotate circuits over - time to avoid some profiling attacks. - - To build a circuit, we choose all the nodes we want to use, and then - construct the circuit. Sometimes, when we want a circuit that ends at a - given hop, and we have an appropriate unused circuit, we "cannibalize" the - existing circuit and extend it to the new terminus. - - These processes are described in more detail below. - - This document describes Tor's automatic path selection logic only; path - selection can be overridden by a controller (with the EXTENDCIRCUIT and - ATTACHSTREAM commands). Paths constructed through these means may - violate some constraints given below. - -1.1. Terminology - - A "path" is an ordered sequence of nodes, not yet built as a circuit. - - A "clean" circuit is one that has not yet been used for any traffic. - - A "fast" or "stable" or "valid" node is one that has the 'Fast' or - 'Stable' or 'Valid' flag - set respectively, based on our current directory information. A "fast" - or "stable" circuit is one consisting only of "fast" or "stable" nodes. - - In an "exit" circuit, the final node is chosen based on waiting stream - requests if any, and in any case it avoids nodes with exit policy of - "reject *:*". An "internal" circuit, on the other hand, is one where - the final node is chosen just like a middle node (ignoring its exit - policy). - - A "request" is a client-side stream or DNS resolve that needs to be - served by a circuit. - - A "pending" circuit is one that we have started to build, but which has - not yet completed. - - A circuit or path "supports" a request if it is okay to use the - circuit/path to fulfill the request, according to the rules given below. - A circuit or path "might support" a request if some aspect of the request - is unknown (usually its target IP), but we believe the path probably - supports the request according to the rules given below. - -2. Building circuits - -2.1. When we build - -2.1.1. Clients build circuits preemptively - - When running as a client, Tor tries to maintain at least a certain - number of clean circuits, so that new streams can be handled - quickly. To increase the likelihood of success, Tor tries to - predict what circuits will be useful by choosing from among nodes - that support the ports we have used in the recent past (by default - one hour). Specifically, on startup Tor tries to maintain one clean - fast exit circuit that allows connections to port 80, and at least - two fast clean stable internal circuits in case we get a resolve - request or hidden service request (at least three if we _run_ a - hidden service). - - After that, Tor will adapt the circuits that it preemptively builds - based on the requests it sees from the user: it tries to have two fast - clean exit circuits available for every port seen within the past hour - (each circuit can be adequate for many predicted ports -- it doesn't - need two separate circuits for each port), and it tries to have the - above internal circuits available if we've seen resolves or hidden - service activity within the past hour. If there are 12 or more clean - circuits open, it doesn't open more even if it has more predictions. - - Only stable circuits can "cover" a port that is listed in the - LongLivedPorts config option. Similarly, hidden service requests - to ports listed in LongLivedPorts make us create stable internal - circuits. - - Note that if there are no requests from the user for an hour, Tor - will predict no use and build no preemptive circuits. - - The Tor client SHOULD NOT store its list of predicted requests to a - persistent medium. - -2.1.2. Clients build circuits on demand - - Additionally, when a client request exists that no circuit (built or - pending) might support, we create a new circuit to support the request. - For exit connections, we pick an exit node that will handle the - most pending requests (choosing arbitrarily among ties), launch a - circuit to end there, and repeat until every unattached request - might be supported by a pending or built circuit. For internal - circuits, we pick an arbitrary acceptable path, repeating as needed. - - In some cases we can reuse an already established circuit if it's - clean; see Section 2.3 (cannibalizing circuits) for details. - -2.1.3. Servers build circuits for testing reachability and bandwidth - - Tor servers test reachability of their ORPort once they have - successfully built a circuit (on start and whenever their IP address - changes). They build an ordinary fast internal circuit with themselves - as the last hop. As soon as any testing circuit succeeds, the Tor - server decides it's reachable and is willing to publish a descriptor. - - We launch multiple testing circuits (one at a time), until we - have NUM_PARALLEL_TESTING_CIRC (4) such circuits open. Then we - do a "bandwidth test" by sending a certain number of relay drop - cells down each circuit: BandwidthRate * 10 / CELL_NETWORK_SIZE - total cells divided across the four circuits, but never more than - CIRCWINDOW_START (1000) cells total. This exercises both outgoing and - incoming bandwidth, and helps to jumpstart the observed bandwidth - (see dir-spec.txt). - - Tor servers also test reachability of their DirPort once they have - established a circuit, but they use an ordinary exit circuit for - this purpose. - -2.1.4. Hidden-service circuits - - See section 4 below. - -2.1.5. Rate limiting of failed circuits - - If we fail to build a circuit N times in a X second period (see Section - 2.3 for how this works), we stop building circuits until the X seconds - have elapsed. - XXXX - -2.1.6. When to tear down circuits - - XXXX - -2.2. Path selection and constraints - - We choose the path for each new circuit before we build it. We choose the - exit node first, followed by the other nodes in the circuit. All paths - we generate obey the following constraints: - - We do not choose the same router twice for the same path. - - We do not choose any router in the same family as another in the same - path. - - We do not choose more than one router in a given /16 subnet - (unless EnforceDistinctSubnets is 0). - - We don't choose any non-running or non-valid router unless we have - been configured to do so. By default, we are configured to allow - non-valid routers in "middle" and "rendezvous" positions. - - If we're using Guard nodes, the first node must be a Guard (see 5 - below) - - XXXX Choosing the length - - For circuits that do not need to be "fast", when choosing among - multiple candidates for a path element, we choose randomly. - - For "fast" circuits, we pick a given router as an exit with probability - proportional to its advertised bandwidth [the smaller of the 'rate' and - 'observed' arguments to the "bandwidth" element in its descriptor]. If a - router's advertised bandwidth is greater than MAX_BELIEVABLE_BANDWIDTH - (currently 10 MB/s), we clip to that value. - - For non-exit positions on "fast" circuits, we pick routers as above, but - we weight the clipped advertised bandwidth of Exit-flagged nodes depending - on the fraction of bandwidth available from non-Exit nodes. Call the - total clipped advertised bandwidth for Exit nodes under consideration E, - and the total clipped advertised bandwidth for all nodes under - consideration T. If E..exit, the request is rewritten to a request for - , and the request is only supported by the exit whose nickname - or fingerprint is . - -2.3. Cannibalizing circuits - - If we need a circuit and have a clean one already established, in - some cases we can adapt the clean circuit for our new - purpose. Specifically, - - For hidden service interactions, we can "cannibalize" a clean internal - circuit if one is available, so we don't need to build those circuits - from scratch on demand. - - We can also cannibalize clean circuits when the client asks to exit - at a given node -- either via the ".exit" notation or because the - destination is running at the same location as an exit node. - - -2.4. Handling failure - - If an attempt to extend a circuit fails (either because the first create - failed or a subsequent extend failed) then the circuit is torn down and is - no longer pending. (XXXX really?) Requests that might have been - supported by the pending circuit thus become unsupported, and a new - circuit needs to be constructed. - - If a stream "begin" attempt fails with an EXITPOLICY error, we - decide that the exit node's exit policy is not correctly advertised, - so we treat the exit node as if it were a non-exit until we retrieve - a fresh descriptor for it. - - XXXX - -3. Attaching streams to circuits - - When a circuit that might support a request is built, Tor tries to attach - the request's stream to the circuit and sends a BEGIN, BEGIN_DIR, - or RESOLVE relay - cell as appropriate. If the request completes unsuccessfully, Tor - considers the reason given in the CLOSE relay cell. [XXX yes, and?] - - - After a request has remained unattached for SocksTimeout (2 minutes - by default), Tor abandons the attempt and signals an error to the - client as appropriate (e.g., by closing the SOCKS connection). - - XXX Timeouts and when Tor auto-retries. - * What stream-end-reasons are appropriate for retrying. - - If no reply to BEGIN/RESOLVE, then the stream will timeout and fail. - -4. Hidden-service related circuits - - XXX Tracking expected hidden service use (client-side and hidserv-side) - -5. Guard nodes - - We use Guard nodes (also called "helper nodes" in the literature) to - prevent certain profiling attacks. Here's the risk: if we choose entry and - exit nodes at random, and an attacker controls C out of N servers - (ignoring advertised bandwidth), then the - attacker will control the entry and exit node of any given circuit with - probability (C/N)^2. But as we make many different circuits over time, - then the probability that the attacker will see a sample of about (C/N)^2 - of our traffic goes to 1. Since statistical sampling works, the attacker - can be sure of learning a profile of our behavior. - - If, on the other hand, we picked an entry node and held it fixed, we would - have probability C/N of choosing a bad entry and being profiled, and - probability (N-C)/N of choosing a good entry and not being profiled. - - When guard nodes are enabled, Tor maintains an ordered list of entry nodes - as our chosen guards, and stores this list persistently to disk. If a Guard - node becomes unusable, rather than replacing it, Tor adds new guards to the - end of the list. When choosing the first hop of a circuit, Tor - chooses at - random from among the first NumEntryGuards (default 3) usable guards on the - list. If there are not at least 2 usable guards on the list, Tor adds - routers until there are, or until there are no more usable routers to add. - - A guard is unusable if any of the following hold: - - it is not marked as a Guard by the networkstatuses, - - it is not marked Valid (and the user hasn't set AllowInvalid entry) - - it is not marked Running - - Tor couldn't reach it the last time it tried to connect - - A guard is unusable for a particular circuit if any of the rules for path - selection in 2.2 are not met. In particular, if the circuit is "fast" - and the guard is not Fast, or if the circuit is "stable" and the guard is - not Stable, or if the guard has already been chosen as the exit node in - that circuit, Tor can't use it as a guard node for that circuit. - - If the guard is excluded because of its status in the networkstatuses for - over 30 days, Tor removes it from the list entirely, preserving order. - - If Tor fails to connect to an otherwise usable guard, it retries - periodically: every hour for six hours, every 4 hours for 3 days, every - 18 hours for a week, and every 36 hours thereafter. Additionally, Tor - retries unreachable guards the first time it adds a new guard to the list, - since it is possible that the old guards were only marked as unreachable - because the network was unreachable or down. - - Tor does not add a guard persistently to the list until the first time we - have connected to it successfully. - -6. Router descriptor purposes - - There are currently three "purposes" supported for router descriptors: - general, controller, and bridge. Most descriptors are of type general - -- these are the ones listed in the consensus, and the ones fetched - and used in normal cases. - - Controller-purpose descriptors are those delivered by the controller - and labelled as such: they will be kept around (and expire like - normal descriptors), and they can be used by the controller in its - CIRCUITEXTEND commands. Otherwise they are ignored by Tor when it - chooses paths. - - Bridge-purpose descriptors are for routers that are used as bridges. See - doc/design-paper/blocking.pdf for more design explanation, or proposal - 125 for specific details. Currently bridge descriptors are used in place - of normal entry guards, for Tor clients that have UseBridges enabled. - - -X. Old notes - -X.1. Do we actually do this? - -How to deal with network down. - - While all helpers are down/unreachable and there are no established - or on-the-way testing circuits, launch a testing circuit. (Do this - periodically in the same way we try to establish normal circuits - when things are working normally.) - (Testing circuits are a special type of circuit, that streams won't - attach to by accident.) - - When a testing circuit succeeds, mark all helpers up and hold - the testing circuit open. - - If a connection to a helper succeeds, close all testing circuits. - Else mark that helper down and try another. - - If the last helper is marked down and we already have a testing - circuit established, then add the first hop of that testing circuit - to the end of our helper node list, close that testing circuit, - and go back to square one. (Actually, rather than closing the - testing circuit, can we get away with converting it to a normal - circuit and beginning to use it immediately?) - - [Do we actually do any of the above? If so, let's spec it. If not, let's - remove it. -NM] - -X.2. A thing we could do to deal with reachability. - -And as a bonus, it leads to an answer to Nick's attack ("If I pick -my helper nodes all on 18.0.0.0:*, then I move, you'll know where I -bootstrapped") -- the answer is to pick your original three helper nodes -without regard for reachability. Then the above algorithm will add some -more that are reachable for you, and if you move somewhere, it's more -likely (though not certain) that some of the originals will become useful. -Is that smart or just complex? - -X.3. Some stuff that worries me about entry guards. 2006 Jun, Nickm. - - It is unlikely for two users to have the same set of entry guards. - Observing a user is sufficient to learn its entry guards. So, as we move - around, entry guards make us linkable. If we want to change guards when - our location (IP? subnet?) changes, we have two bad options. We could - - Drop the old guards. But if we go back to our old location, - we'll not use our old guards. For a laptop that sometimes gets used - from work and sometimes from home, this is pretty fatal. - - Remember the old guards as associated with the old location, and use - them again if we ever go back to the old location. This would be - nasty, since it would force us to record where we've been. - - [Do we do any of this now? If not, this should move into 099-misc or - 098-todo. -NM] - diff --git a/doc/spec/proposals/000-index.txt b/doc/spec/proposals/000-index.txt deleted file mode 100644 index d75157650..000000000 --- a/doc/spec/proposals/000-index.txt +++ /dev/null @@ -1,161 +0,0 @@ -Filename: 000-index.txt -Title: Index of Tor Proposals -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 26-Jan-2007 -Status: Meta - -Overview: - - This document provides an index to Tor proposals. - - This is an informational document. - - Everything in this document below the line of '=' signs is automatically - generated by reindex.py; do not edit by hand. - -============================================================ -Proposals by number: - -000 Index of Tor Proposals [META] -001 The Tor Proposal Process [META] -098 Proposals that should be written [META] -099 Miscellaneous proposals [META] -100 Tor Unreliable Datagram Extension Proposal [DEAD] -101 Voting on the Tor Directory System [CLOSED] -102 Dropping "opt" from the directory format [CLOSED] -103 Splitting identity key from regularly used signing key [CLOSED] -104 Long and Short Router Descriptors [CLOSED] -105 Version negotiation for the Tor protocol [CLOSED] -106 Checking fewer things during TLS handshakes [CLOSED] -107 Uptime Sanity Checking [CLOSED] -108 Base "Stable" Flag on Mean Time Between Failures [CLOSED] -109 No more than one server per IP address [CLOSED] -110 Avoiding infinite length circuits [ACCEPTED] -111 Prioritizing local traffic over relayed traffic [CLOSED] -112 Bring Back Pathlen Coin Weight [SUPERSEDED] -113 Simplifying directory authority administration [SUPERSEDED] -114 Distributed Storage for Tor Hidden Service Descriptors [CLOSED] -115 Two Hop Paths [DEAD] -116 Two hop paths from entry guards [DEAD] -117 IPv6 exits [ACCEPTED] -118 Advertising multiple ORPorts at once [ACCEPTED] -119 New PROTOCOLINFO command for controllers [CLOSED] -120 Shutdown descriptors when Tor servers stop [DEAD] -121 Hidden Service Authentication [FINISHED] -122 Network status entries need a new Unnamed flag [CLOSED] -123 Naming authorities automatically create bindings [CLOSED] -124 Blocking resistant TLS certificate usage [SUPERSEDED] -125 Behavior for bridge users, bridge relays, and bridge authorities [CLOSED] -126 Getting GeoIP data and publishing usage summaries [CLOSED] -127 Relaying dirport requests to Tor download site / website [DRAFT] -128 Families of private bridges [DEAD] -129 Block Insecure Protocols by Default [CLOSED] -130 Version 2 Tor connection protocol [CLOSED] -131 Help users to verify they are using Tor [NEEDS-REVISION] -132 A Tor Web Service For Verifying Correct Browser Configuration [DRAFT] -133 Incorporate Unreachable ORs into the Tor Network [DRAFT] -134 More robust consensus voting with diverse authority sets [ACCEPTED] -135 Simplify Configuration of Private Tor Networks [CLOSED] -136 Mass authority migration with legacy keys [CLOSED] -137 Keep controllers informed as Tor bootstraps [CLOSED] -138 Remove routers that are not Running from consensus documents [CLOSED] -139 Download consensus documents only when it will be trusted [CLOSED] -140 Provide diffs between consensuses [ACCEPTED] -141 Download server descriptors on demand [DRAFT] -142 Combine Introduction and Rendezvous Points [DEAD] -143 Improvements of Distributed Storage for Tor Hidden Service Descriptors [OPEN] -144 Increase the diversity of circuits by detecting nodes belonging the same provider [DRAFT] -145 Separate "suitable as a guard" from "suitable as a new guard" [OPEN] -146 Add new flag to reflect long-term stability [OPEN] -147 Eliminate the need for v2 directories in generating v3 directories [ACCEPTED] -148 Stream end reasons from the client side should be uniform [CLOSED] -149 Using data from NETINFO cells [OPEN] -150 Exclude Exit Nodes from a circuit [CLOSED] -151 Improving Tor Path Selection [DRAFT] -152 Optionally allow exit from single-hop circuits [CLOSED] -153 Automatic software update protocol [SUPERSEDED] -154 Automatic Software Update Protocol [SUPERSEDED] -155 Four Improvements of Hidden Service Performance [FINISHED] -156 Tracking blocked ports on the client side [OPEN] -157 Make certificate downloads specific [ACCEPTED] -158 Clients download consensus + microdescriptors [OPEN] -159 Exit Scanning [OPEN] - - -Proposals by status: - - DRAFT: - 127 Relaying dirport requests to Tor download site / website - 132 A Tor Web Service For Verifying Correct Browser Configuration - 133 Incorporate Unreachable ORs into the Tor Network - 141 Download server descriptors on demand - 144 Increase the diversity of circuits by detecting nodes belonging the same provider - 151 Improving Tor Path Selection - NEEDS-REVISION: - 131 Help users to verify they are using Tor - OPEN: - 143 Improvements of Distributed Storage for Tor Hidden Service Descriptors [for 0.2.1.x] - 145 Separate "suitable as a guard" from "suitable as a new guard" [for 0.2.1.x] - 146 Add new flag to reflect long-term stability [for 0.2.1.x] - 149 Using data from NETINFO cells [for 0.2.1.x] - 156 Tracking blocked ports on the client side [for 0.2.?] - 158 Clients download consensus + microdescriptors - 159 Exit Scanning - ACCEPTED: - 110 Avoiding infinite length circuits [for 0.2.1.x] [in 0.2.1.3-alpha] - 117 IPv6 exits [for 0.2.1.x] - 118 Advertising multiple ORPorts at once [for 0.2.1.x] - 134 More robust consensus voting with diverse authority sets [for 0.2.2.x] - 140 Provide diffs between consensuses [for 0.2.2.x] - 147 Eliminate the need for v2 directories in generating v3 directories [for 0.2.1.x] - 157 Make certificate downloads specific [for 0.2.1.x] - META: - 000 Index of Tor Proposals - 001 The Tor Proposal Process - 098 Proposals that should be written - 099 Miscellaneous proposals - FINISHED: - 121 Hidden Service Authentication [in 0.2.1.x] - 155 Four Improvements of Hidden Service Performance [in 0.2.1.x] - CLOSED: - 101 Voting on the Tor Directory System [in 0.2.0.x] - 102 Dropping "opt" from the directory format [in 0.2.0.x] - 103 Splitting identity key from regularly used signing key [in 0.2.0.x] - 104 Long and Short Router Descriptors [in 0.2.0.x] - 105 Version negotiation for the Tor protocol [in 0.2.0.x] - 106 Checking fewer things during TLS handshakes [in 0.2.0.x] - 107 Uptime Sanity Checking [in 0.2.0.x] - 108 Base "Stable" Flag on Mean Time Between Failures [in 0.2.0.x] - 109 No more than one server per IP address [in 0.2.0.x] - 111 Prioritizing local traffic over relayed traffic [in 0.2.0.x] - 114 Distributed Storage for Tor Hidden Service Descriptors [in 0.2.0.x] - 119 New PROTOCOLINFO command for controllers [in 0.2.0.x] - 122 Network status entries need a new Unnamed flag [in 0.2.0.x] - 123 Naming authorities automatically create bindings [in 0.2.0.x] - 125 Behavior for bridge users, bridge relays, and bridge authorities [in 0.2.0.x] - 126 Getting GeoIP data and publishing usage summaries [in 0.2.0.x] - 129 Block Insecure Protocols by Default [in 0.2.0.x] - 130 Version 2 Tor connection protocol [in 0.2.0.x] - 135 Simplify Configuration of Private Tor Networks [for 0.2.1.x] [in 0.2.1.2-alpha] - 136 Mass authority migration with legacy keys [in 0.2.0.x] - 137 Keep controllers informed as Tor bootstraps [in 0.2.1.x] - 138 Remove routers that are not Running from consensus documents [in 0.2.1.2-alpha] - 139 Download consensus documents only when it will be trusted [in 0.2.1.x] - 148 Stream end reasons from the client side should be uniform [in 0.2.1.9-alpha] - 150 Exclude Exit Nodes from a circuit [in 0.2.1.3-alpha] - 152 Optionally allow exit from single-hop circuits [in 0.2.1.6-alpha] - SUPERSEDED: - 112 Bring Back Pathlen Coin Weight - 113 Simplifying directory authority administration - 124 Blocking resistant TLS certificate usage - 153 Automatic software update protocol - 154 Automatic Software Update Protocol - DEAD: - 100 Tor Unreliable Datagram Extension Proposal - 115 Two Hop Paths - 116 Two hop paths from entry guards - 120 Shutdown descriptors when Tor servers stop - 128 Families of private bridges - 142 Combine Introduction and Rendezvous Points diff --git a/doc/spec/proposals/001-process.txt b/doc/spec/proposals/001-process.txt deleted file mode 100644 index 3a767b5fa..000000000 --- a/doc/spec/proposals/001-process.txt +++ /dev/null @@ -1,187 +0,0 @@ -Filename: 001-process.txt -Title: The Tor Proposal Process -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 30-Jan-2007 -Status: Meta - -Overview: - - This document describes how to change the Tor specifications, how Tor - proposals work, and the relationship between Tor proposals and the - specifications. - - This is an informational document. - -Motivation: - - Previously, our process for updating the Tor specifications was maximally - informal: we'd patch the specification (sometimes forking first, and - sometimes not), then discuss the patches, reach consensus, and implement - the changes. - - This had a few problems. - - First, even at its most efficient, the old process would often have the - spec out of sync with the code. The worst cases were those where - implementation was deferred: the spec and code could stay out of sync for - versions at a time. - - Second, it was hard to participate in discussion, since you had to know - which portions of the spec were a proposal, and which were already - implemented. - - Third, it littered the specifications with too many inline comments. - [This was a real problem -NM] - [Especially when it went to multiple levels! -NM] - [XXXX especially when they weren't signed and talked about that - thing that you can't remember after a year] - -How to change the specs now: - - First, somebody writes a proposal document. It should describe the change - that should be made in detail, and give some idea of how to implement it. - Once it's fleshed out enough, it becomes a proposal. - - Like an RFC, every proposal gets a number. Unlike RFCs, proposals can - change over time and keep the same number, until they are finally - accepted or rejected. The history for each proposal - will be stored in the Tor Subversion repository. - - Once a proposal is in the repository, we should discuss and improve it - until we've reached consensus that it's a good idea, and that it's - detailed enough to implement. When this happens, we implement the - proposal and incorporate it into the specifications. Thus, the specs - remain the canonical documentation for the Tor protocol: no proposal is - ever the canonical documentation for an implemented feature. - - (This process is pretty similar to the Python Enhancement Process, with - the major exception that Tor proposals get re-integrated into the specs - after implementation, whereas PEPs _become_ the new spec.) - - {It's still okay to make small changes directly to the spec if the code - can be - written more or less immediately, or cosmetic changes if no code change is - required. This document reflects the current developers' _intent_, not - a permanent promise to always use this process in the future: we reserve - the right to get really excited and run off and implement something in a - caffeine-or-m&m-fueled all-night hacking session.} - -How new proposals get added: - - Once an idea has been proposed on the development list, a properly formatted - (see below) draft exists, and rough consensus within the active development - community exists that this idea warrants consideration, the proposal editor - will officially add the proposal. - - To get your proposal in, send it to or-dev. - - The current proposal editor is Nick Mathewson. - -What should go in a proposal: - - Every proposal should have a header containing these fields: - Filename, Title, Version, Last-Modified, Author, Created, Status. - The Version and Last-Modified fields should use the SVN Revision and Date - tags respectively. - - These fields are optional but recommended: - Target, Implemented-In. - The Target field should describe which version the proposal is hoped to be - implemented in (if it's Open or Accepted). The Implemented-In field - should describe which version the proposal was implemented in (if it's - Finished or Closed). - - The body of the proposal should start with an Overview section explaining - what the proposal's about, what it does, and about what state it's in. - - After the Overview, the proposal becomes more free-form. Depending on its - the length and complexity, the proposal can break into sections as - appropriate, or follow a short discursive format. Every proposal should - contain at least the following information before it is "ACCEPTED", - though the information does not need to be in sections with these names. - - Motivation: What problem is the proposal trying to solve? Why does - this problem matter? If several approaches are possible, why take this - one? - - Design: A high-level view of what the new or modified features are, how - the new or modified features work, how they interoperate with each - other, and how they interact with the rest of Tor. This is the main - body of the proposal. Some proposals will start out with only a - Motivation and a Design, and wait for a specification until the - Design seems approximately right. - - Security implications: What effects the proposed changes might have on - anonymity, how well understood these effects are, and so on. - - Specification: A detailed description of what needs to be added to the - Tor specifications in order to implement the proposal. This should - be in about as much detail as the specifications will eventually - contain: it should be possible for independent programmers to write - mutually compatible implementations of the proposal based on its - specifications. - - Compatibility: Will versions of Tor that follow the proposal be - compatible with versions that do not? If so, how will compatibility - be achieved? Generally, we try to not drop compatibility if at - all possible; we haven't made a "flag day" change since May 2004, - and we don't want to do another one. - - Implementation: If the proposal will be tricky to implement in Tor's - current architecture, the document can contain some discussion of how - to go about making it work. - - Performance and scalability notes: If the feature will have an effect - on performance (in RAM, CPU, bandwidth) or scalability, there should - be some analysis on how significant this effect will be, so that we - can avoid really expensive performance regressions, and so we can - avoid wasting time on insignificant gains. - -Proposal status: - - Open: A proposal under discussion. - - Accepted: The proposal is complete, and we intend to implement it. - After this point, substantive changes to the proposal should be - avoided, and regarded as a sign of the process having failed - somewhere. - - Finished: The proposal has been accepted and implemented. After this - point, the proposal should not be changed. - - Closed: The proposal has been accepted, implemented, and merged into the - main specification documents. The proposal should not be changed after - this point. - - Rejected: We're not going to implement the feature as described here, - though we might do some other version. See comments in the document - for details. The proposal should not be changed after this point; - to bring up some other version of the idea, write a new proposal. - - Draft: This isn't a complete proposal yet; there are definite missing - pieces. Please don't add any new proposals with this status; put them - in the "ideas" sub-directory instead. - - Needs-Revision: The idea for the proposal is a good one, but the proposal - as it stands has serious problems that keep it from being accepted. - See comments in the document for details. - - Dead: The proposal hasn't been touched in a long time, and it doesn't look - like anybody is going to complete it soon. It can become "Open" again - if it gets a new proponent. - - Needs-Research: There are research problems that need to be solved before - it's clear whether the proposal is a good idea. - - Meta: This is not a proposal, but a document about proposals. - - - The editor maintains the correct status of proposals, based on rough - consensus and his own discretion. - -Proposal numbering: - - Numbers 000-099 are reserved for special and meta-proposals. 100 and up - are used for actual proposals. Numbers aren't recycled. diff --git a/doc/spec/proposals/098-todo.txt b/doc/spec/proposals/098-todo.txt deleted file mode 100644 index e891ea890..000000000 --- a/doc/spec/proposals/098-todo.txt +++ /dev/null @@ -1,109 +0,0 @@ -Filename: 098-todo.txt -Title: Proposals that should be written -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson, Roger Dingledine -Created: 26-Jan-2007 -Status: Meta - -Overview: - - This document lists ideas that various people have had for improving the - Tor protocol. These should be implemented and specified if they're - trivial, or written up as proposals if they're not. - - This is an active document, to be edited as proposals are written and as - we come up with new ideas for proposals. We should take stuff out as it - seems irrelevant. - - -For some later protocol version. - - - It would be great to get smarter about identity and linkability. - It's not crazy to say, "Never use the same circuit for my SSH - connections and my web browsing." How far can/should we take this? - See ideas/xxx-separate-streams-by-port.txt for a start. - - - Fix onionskin handshake scheme to be more mainstream, less nutty. - Can we just do - E(HMAC(g^x), g^x) rather than just E(g^x) ? - No, that has the same flaws as before. We should send - E(g^x, C) with random C and expect g^y, HMAC_C(K=g^xy). - Better ask Ian; probably Stephen too. - - - Length on CREATE and friends - - - Versioning on circuits and create cells, so we have a clear path - to improve the circuit protocol. - - - SHA1 is showing its age. We should get a design for upgrading our - hash once the AHS competition is done, or even sooner. - - - Not being able to upgrade ciphersuites or increase key lengths is - lame. - - Paul has some ideas about circuit creation; read his PET paper once it's - out. - -Any time: - - - Some ideas for revising the directory protocol: - - Extend the "r" line in network-status to give a set of buckets (say, - comma-separated) for that router. - - Buckets are deterministic based on IP address. - - Then clients can choose a bucket (or set of buckets) to - download and use. - - We need a way for the authorities to declare that nodes are in a - family. Also, it kinda sucks that family declarations use O(N^2) space - in the descriptors. - - REASON_CONNECTFAILED should include an IP. - - Spec should incorporate some prose from tor-design to be more readable. - - Spec when we should rotate which keys - - Spec how to publish descriptors less often - - Describe pros and cons of non-deterministic path lengths - - - We should use a variable-length path length by default -- 3 +/- some - distribution. Need to think harder about allowing values less than 3, - and there's a tradeoff between having a wide variance and performance. - - - Clients currently use certs during TLS. Is this wise? It does make it - easier for servers to tell which NATted client is which. We could use a - seprate set of certs for each guard, I suppose, but generating so many - certs could get expensive. Omitting them entirely would make OP->OR - easier to tell from OR->OR. - -Things that should change... - -B.1. ... but which will require backward-incompatible change - - - Circuit IDs should be longer. - . IPv6 everywhere. - - Maybe, keys should be longer. - - Maybe, key-length should be adjustable. How to do this without - making anonymity suck? - - Drop backward compatibility. - - We should use a 128-bit subgroup of our DH prime. - - Handshake should use HMAC. - - Multiple cell lengths. - - Ability to split circuits across paths (If this is useful.) - - SENDME windows should be dynamic. - - - Directory - - Stop ever mentioning socks ports - -B.1. ... and that will require no changes - - - Advertised outbound IP? - - Migrate streams across circuits. - - Fix bug 469 by limiting the number of simultaneous connections per IP. - -B.2. ... and that we have no idea how to do. - - - UDP (as transport) - - UDP (as content) - - Use a better AES mode that has built-in integrity checking, - doesn't grow with the number of hops, is not patented, and - is implemented and maintained by smart people. - -Let onion keys be not just RSA but maybe DH too, for Paul's reply onion -design. - diff --git a/doc/spec/proposals/099-misc.txt b/doc/spec/proposals/099-misc.txt deleted file mode 100644 index ba13ea2a7..000000000 --- a/doc/spec/proposals/099-misc.txt +++ /dev/null @@ -1,30 +0,0 @@ -Filename: 099-misc.txt -Title: Miscellaneous proposals -Version: $Revision$ -Last-Modified: $Date$ -Author: Various -Created: 26-Jan-2007 -Status: Meta - -Overview: - - This document is for small proposal ideas that are about one paragraph in - length. From here, ideas can be rejected outright, expanded into full - proposals, or specified and implemented as-is. - -Proposals - -1. Directory compression. - - Gzip would be easier to work with than zlib; bzip2 would result in smaller - data lengths. [Concretely, we're looking at about 10-15% space savings at - the expense of 3-5x longer compression time for using bzip2.] Doing - on-the-fly gzip requires zlib 1.2 or later; doing bzip2 requires bzlib. - Pre-compressing status documents in multiple formats would force us to use - more memory to hold them. - - Status: Open - - -- Nick Mathewson - - diff --git a/doc/spec/proposals/100-tor-spec-udp.txt b/doc/spec/proposals/100-tor-spec-udp.txt deleted file mode 100644 index 8224682ec..000000000 --- a/doc/spec/proposals/100-tor-spec-udp.txt +++ /dev/null @@ -1,424 +0,0 @@ -Filename: 100-tor-spec-udp.txt -Title: Tor Unreliable Datagram Extension Proposal -Version: $Revision$ -Last-Modified: $Date$ -Author: Marc Liberatore -Created: 23 Feb 2006 -Status: Dead - -Overview: - - This is a modified version of the Tor specification written by Marc - Liberatore to add UDP support to Tor. For each TLS link, it adds a - corresponding DTLS link: control messages and TCP data flow over TLS, and - UDP data flows over DTLS. - - This proposal is not likely to be accepted as-is; see comments at the end - of the document. - - -Contents - -0. Introduction - - Tor is a distributed overlay network designed to anonymize low-latency - TCP-based applications. The current tor specification supports only - TCP-based traffic. This limitation prevents the use of tor to anonymize - other important applications, notably voice over IP software. This document - is a proposal to extend the tor specification to support UDP traffic. - - The basic design philosophy of this extension is to add support for - tunneling unreliable datagrams through tor with as few modifications to the - protocol as possible. As currently specified, tor cannot directly support - such tunneling, as connections between nodes are built using transport layer - security (TLS) atop TCP. The latency incurred by TCP is likely unacceptable - to the operation of most UDP-based application level protocols. - - Thus, we propose the addition of links between nodes using datagram - transport layer security (DTLS). These links allow packets to traverse a - route through tor quickly, but their unreliable nature requires minor - changes to the tor protocol. This proposal outlines the necessary - additions and changes to the tor specification to support UDP traffic. - - We note that a separate set of DTLS links between nodes creates a second - overlay, distinct from the that composed of TLS links. This separation and - resulting decrease in each anonymity set's size will make certain attacks - easier. However, it is our belief that VoIP support in tor will - dramatically increase its appeal, and correspondingly, the size of its user - base, number of deployed nodes, and total traffic relayed. These increases - should help offset the loss of anonymity that two distinct networks imply. - -1. Overview of Tor-UDP and its complications - - As described above, this proposal extends the Tor specification to support - UDP with as few changes as possible. Tor's overlay network is managed - through TLS based connections; we will re-use this control plane to set up - and tear down circuits that relay UDP traffic. These circuits be built atop - DTLS, in a fashion analogous to how Tor currently sends TCP traffic over - TLS. - - The unreliability of DTLS circuits creates problems for Tor at two levels: - - 1. Tor's encryption of the relay layer does not allow independent - decryption of individual records. If record N is not received, then - record N+1 will not decrypt correctly, as the counter for AES/CTR is - maintained implicitly. - - 2. Tor's end-to-end integrity checking works under the assumption that - all RELAY cells are delivered. This assumption is invalid when cells - are sent over DTLS. - - The fix for the first problem is straightforward: add an explicit sequence - number to each cell. To fix the second problem, we introduce a - system of nonces and hashes to RELAY packets. - - In the following sections, we mirror the layout of the Tor Protocol - Specification, presenting the necessary modifications to the Tor protocol as - a series of deltas. - -2. Connections - - Tor-UDP uses DTLS for encryption of some links. All DTLS links must have - corresponding TLS links, as all control messages are sent over TLS. All - implementations MUST support the DTLS ciphersuite "[TODO]". - - DTLS connections are formed using the same protocol as TLS connections. - This occurs upon request, following a CREATE_UDP or CREATE_FAST_UDP cell, - as detailed in section 4.6. - - Once a paired TLS/DTLS connection is established, the two sides send cells - to one another. All but two types of cells are sent over TLS links. RELAY - cells containing the commands RELAY_UDP_DATA and RELAY_UDP_DROP, specified - below, are sent over DTLS links. [Should all cells still be 512 bytes long? - Perhaps upon completion of a preliminary implementation, we should do a - performance evaluation for some class of UDP traffic, such as VoIP. - ML] - Cells may be sent embedded in TLS or DTLS records of any size or divided - across such records. The framing of these records MUST NOT leak any more - information than the above differentiation on the basis of cell type. [I am - uncomfortable with this leakage, but don't see any simple, elegant way - around it. -ML] - - As with TLS connections, DTLS connections are not permanent. - -3. Cell format - - Each cell contains the following fields: - - CircID [2 bytes] - Command [1 byte] - Sequence Number [2 bytes] - Payload (padded with 0 bytes) [507 bytes] - [Total size: 512 bytes] - - The 'Command' field holds one of the following values: - 0 -- PADDING (Padding) (See Sec 6.2) - 1 -- CREATE (Create a circuit) (See Sec 4) - 2 -- CREATED (Acknowledge create) (See Sec 4) - 3 -- RELAY (End-to-end data) (See Sec 5) - 4 -- DESTROY (Stop using a circuit) (See Sec 4) - 5 -- CREATE_FAST (Create a circuit, no PK) (See Sec 4) - 6 -- CREATED_FAST (Circuit created, no PK) (See Sec 4) - 7 -- CREATE_UDP (Create a UDP circuit) (See Sec 4) - 8 -- CREATED_UDP (Acknowledge UDP create) (See Sec 4) - 9 -- CREATE_FAST_UDP (Create a UDP circuit, no PK) (See Sec 4) - 10 -- CREATED_FAST_UDP(UDP circuit created, no PK) (See Sec 4) - - The sequence number allows for AES/CTR decryption of RELAY cells - independently of one another; this functionality is required to support - cells sent over DTLS. The sequence number is described in more detail in - section 4.5. - - [Should the sequence number only appear in RELAY packets? The overhead is - small, and I'm hesitant to force more code paths on the implementor. -ML] - [There's already a separate relay header that has other material in it, - so it wouldn't be the end of the world to move it there if it's - appropriate. -RD] - - [Having separate commands for UDP circuits seems necessary, unless we can - assume a flag day event for a large number of tor nodes. -ML] - -4. Circuit management - -4.2. Setting circuit keys - - Keys are set up for UDP circuits in the same fashion as for TCP circuits. - Each UDP circuit shares keys with its corresponding TCP circuit. - - [If the keys are used for both TCP and UDP connections, how does it - work to mix sequence-number-less cells with sequenced-numbered cells -- - how do you know you have the encryption order right? -RD] - -4.3. Creating circuits - - UDP circuits are created as TCP circuits, using the *_UDP cells as - appropriate. - -4.4. Tearing down circuits - - UDP circuits are torn down as TCP circuits, using the *_UDP cells as - appropriate. - -4.5. Routing relay cells - - When an OR receives a RELAY cell, it checks the cell's circID and - determines whether it has a corresponding circuit along that - connection. If not, the OR drops the RELAY cell. - - Otherwise, if the OR is not at the OP edge of the circuit (that is, - either an 'exit node' or a non-edge node), it de/encrypts the payload - with AES/CTR, as follows: - 'Forward' relay cell (same direction as CREATE): - Use Kf as key; decrypt, using sequence number to synchronize - ciphertext and keystream. - 'Back' relay cell (opposite direction from CREATE): - Use Kb as key; encrypt, using sequence number to synchronize - ciphertext and keystream. - Note that in counter mode, decrypt and encrypt are the same operation. - [Since the sequence number is only 2 bytes, what do you do when it - rolls over? -RD] - - Each stream encrypted by a Kf or Kb has a corresponding unique state, - captured by a sequence number; the originator of each such stream chooses - the initial sequence number randomly, and increments it only with RELAY - cells. [This counts cells; unlike, say, TCP, tor uses fixed-size cells, so - there's no need for counting bytes directly. Right? - ML] - [I believe this is true. You'll find out for sure when you try to - build it. ;) -RD] - - The OR then decides whether it recognizes the relay cell, by - inspecting the payload as described in section 5.1 below. If the OR - recognizes the cell, it processes the contents of the relay cell. - Otherwise, it passes the decrypted relay cell along the circuit if - the circuit continues. If the OR at the end of the circuit - encounters an unrecognized relay cell, an error has occurred: the OR - sends a DESTROY cell to tear down the circuit. - - When a relay cell arrives at an OP, the OP decrypts the payload - with AES/CTR as follows: - OP receives data cell: - For I=N...1, - Decrypt with Kb_I, using the sequence number as above. If the - payload is recognized (see section 5.1), then stop and process - the payload. - - For more information, see section 5 below. - -4.6. CREATE_UDP and CREATED_UDP cells - - Users set up UDP circuits incrementally. The procedure is similar to that - for TCP circuits, as described in section 4.1. In addition to the TLS - connection to the first node, the OP also attempts to open a DTLS - connection. If this succeeds, the OP sends a CREATE_UDP cell, with a - payload in the same format as a CREATE cell. To extend a UDP circuit past - the first hop, the OP sends an EXTEND_UDP relay cell (see section 5) which - instructs the last node in the circuit to send a CREATE_UDP cell to extend - the circuit. - - The relay payload for an EXTEND_UDP relay cell consists of: - Address [4 bytes] - TCP port [2 bytes] - UDP port [2 bytes] - Onion skin [186 bytes] - Identity fingerprint [20 bytes] - - The address field and ports denote the IPV4 address and ports of the next OR - in the circuit. - - The payload for a CREATED_UDP cell or the relay payload for an - RELAY_EXTENDED_UDP cell is identical to that of the corresponding CREATED or - RELAY_EXTENDED cell. Both circuits are established using the same key. - - Note that the existence of a UDP circuit implies the - existence of a corresponding TCP circuit, sharing keys, sequence numbers, - and any other relevant state. - -4.6.1 CREATE_FAST_UDP/CREATED_FAST_UDP cells - - As above, the OP must successfully connect using DTLS before attempting to - send a CREATE_FAST_UDP cell. Otherwise, the procedure is the same as in - section 4.1.1. - -5. Application connections and stream management - -5.1. Relay cells - - Within a circuit, the OP and the exit node use the contents of RELAY cells - to tunnel end-to-end commands, TCP connections ("Streams"), and UDP packets - across circuits. End-to-end commands and UDP packets can be initiated by - either edge; streams are initiated by the OP. - - The payload of each unencrypted RELAY cell consists of: - Relay command [1 byte] - 'Recognized' [2 bytes] - StreamID [2 bytes] - Digest [4 bytes] - Length [2 bytes] - Data [498 bytes] - - The relay commands are: - 1 -- RELAY_BEGIN [forward] - 2 -- RELAY_DATA [forward or backward] - 3 -- RELAY_END [forward or backward] - 4 -- RELAY_CONNECTED [backward] - 5 -- RELAY_SENDME [forward or backward] - 6 -- RELAY_EXTEND [forward] - 7 -- RELAY_EXTENDED [backward] - 8 -- RELAY_TRUNCATE [forward] - 9 -- RELAY_TRUNCATED [backward] - 10 -- RELAY_DROP [forward or backward] - 11 -- RELAY_RESOLVE [forward] - 12 -- RELAY_RESOLVED [backward] - 13 -- RELAY_BEGIN_UDP [forward] - 14 -- RELAY_DATA_UDP [forward or backward] - 15 -- RELAY_EXTEND_UDP [forward] - 16 -- RELAY_EXTENDED_UDP [backward] - 17 -- RELAY_DROP_UDP [forward or backward] - - Commands labelled as "forward" must only be sent by the originator - of the circuit. Commands labelled as "backward" must only be sent by - other nodes in the circuit back to the originator. Commands marked - as either can be sent either by the originator or other nodes. - - The 'recognized' field in any unencrypted relay payload is always set to - zero. - - The 'digest' field can have two meanings. For all cells sent over TLS - connections (that is, all commands and all non-UDP RELAY data), it is - computed as the first four bytes of the running SHA-1 digest of all the - bytes that have been sent reliably and have been destined for this hop of - the circuit or originated from this hop of the circuit, seeded from Df or Db - respectively (obtained in section 4.2 above), and including this RELAY - cell's entire payload (taken with the digest field set to zero). Cells sent - over DTLS connections do not affect this running digest. Each cell sent - over DTLS (that is, RELAY_DATA_UDP and RELAY_DROP_UDP) has the digest field - set to the SHA-1 digest of the current RELAY cells' entire payload, with the - digest field set to zero. Coupled with a randomly-chosen streamID, this - provides per-cell integrity checking on UDP cells. - [If you drop malformed UDP relay cells but don't close the circuit, - then this 8 bytes of digest is not as strong as what we get in the - TCP-circuit side. Is this a problem? -RD] - - When the 'recognized' field of a RELAY cell is zero, and the digest - is correct, the cell is considered "recognized" for the purposes of - decryption (see section 4.5 above). - - (The digest does not include any bytes from relay cells that do - not start or end at this hop of the circuit. That is, it does not - include forwarded data. Therefore if 'recognized' is zero but the - digest does not match, the running digest at that node should - not be updated, and the cell should be forwarded on.) - - All RELAY cells pertaining to the same tunneled TCP stream have the - same streamID. Such streamIDs are chosen arbitrarily by the OP. RELAY - cells that affect the entire circuit rather than a particular - stream use a StreamID of zero. - - All RELAY cells pertaining to the same UDP tunnel have the same streamID. - This streamID is chosen randomly by the OP, but cannot be zero. - - The 'Length' field of a relay cell contains the number of bytes in - the relay payload which contain real payload data. The remainder of - the payload is padded with NUL bytes. - - If the RELAY cell is recognized but the relay command is not - understood, the cell must be dropped and ignored. Its contents - still count with respect to the digests, though. [Before - 0.1.1.10, Tor closed circuits when it received an unknown relay - command. Perhaps this will be more forward-compatible. -RD] - -5.2.1. Opening UDP tunnels and transferring data - - To open a new anonymized UDP connection, the OP chooses an open - circuit to an exit that may be able to connect to the destination - address, selects a random streamID not yet used on that circuit, - and constructs a RELAY_BEGIN_UDP cell with a payload encoding the address - and port of the destination host. The payload format is: - - ADDRESS | ':' | PORT | [00] - - where ADDRESS can be a DNS hostname, or an IPv4 address in - dotted-quad format, or an IPv6 address surrounded by square brackets; - and where PORT is encoded in decimal. - - [What is the [00] for? -NM] - [It's so the payload is easy to parse out with string funcs -RD] - - Upon receiving this cell, the exit node resolves the address as necessary. - If the address cannot be resolved, the exit node replies with a RELAY_END - cell. (See 5.4 below.) Otherwise, the exit node replies with a - RELAY_CONNECTED cell, whose payload is in one of the following formats: - The IPv4 address to which the connection was made [4 octets] - A number of seconds (TTL) for which the address may be cached [4 octets] - or - Four zero-valued octets [4 octets] - An address type (6) [1 octet] - The IPv6 address to which the connection was made [16 octets] - A number of seconds (TTL) for which the address may be cached [4 octets] - [XXXX Versions of Tor before 0.1.1.6 ignore and do not generate the TTL - field. No version of Tor currently generates the IPv6 format.] - - The OP waits for a RELAY_CONNECTED cell before sending any data. - Once a connection has been established, the OP and exit node - package UDP data in RELAY_DATA_UDP cells, and upon receiving such - cells, echo their contents to the corresponding socket. - RELAY_DATA_UDP cells sent to unrecognized streams are dropped. - - Relay RELAY_DROP_UDP cells are long-range dummies; upon receiving such - a cell, the OR or OP must drop it. - -5.3. Closing streams - - UDP tunnels are closed in a fashion corresponding to TCP connections. - -6. Flow Control - - UDP streams are not subject to flow control. - -7.2. Router descriptor format. - -The items' formats are as follows: - "router" nickname address ORPort SocksPort DirPort UDPPort - - Indicates the beginning of a router descriptor. "address" must be - an IPv4 address in dotted-quad format. The last three numbers - indicate the TCP ports at which this OR exposes - functionality. ORPort is a port at which this OR accepts TLS - connections for the main OR protocol; SocksPort is deprecated and - should always be 0; DirPort is the port at which this OR accepts - directory-related HTTP connections; and UDPPort is a port at which - this OR accepts DTLS connections for UDP data. If any port is not - supported, the value 0 is given instead of a port number. - -Other sections: - -What changes need to happen to each node's exit policy to support this? -RD - -Switching to UDP means managing the queues of incoming packets better, -so we don't miss packets. How does this interact with doing large public -key operations (handshakes) in the same thread? -RD - -======================================================================== -COMMENTS -======================================================================== - -[16 May 2006] - -I don't favor this approach; it makes packet traffic partitioned from -stream traffic end-to-end. The architecture I'd like to see is: - - A *All* Tor-to-Tor traffic is UDP/DTLS, unless we need to fall back on - TCP/TLS for firewall penetration or something. (This also gives us an - upgrade path for routing through legacy servers.) - - B Stream traffic is handled with end-to-end per-stream acks/naks and - retries. On failure, the data is retransmitted in a new RELAY_DATA cell; - a cell isn't retransmitted. - -We'll need to do A anyway, to fix our behavior on packet-loss. Once we've -done so, B is more or less inevitable, and we can support end-to-end UDP -traffic "for free". - -(Also, there are some details that this draft spec doesn't address. For -example, what happens when a UDP packet doesn't fit in a single cell?) - --NM diff --git a/doc/spec/proposals/101-dir-voting.txt b/doc/spec/proposals/101-dir-voting.txt deleted file mode 100644 index be900a641..000000000 --- a/doc/spec/proposals/101-dir-voting.txt +++ /dev/null @@ -1,285 +0,0 @@ -Filename: 101-dir-voting.txt -Title: Voting on the Tor Directory System -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: Nov 2006 -Status: Closed -Implemented-In: 0.2.0.x - -Overview - - This document describes a consensus voting scheme for Tor directories; - instead of publishing different network statuses, directories would vote on - and publish a single "consensus" network status document. - - This is an open proposal. - -Proposal: - -0. Scope and preliminaries - - This document describes a consensus voting scheme for Tor directories. - Once it's accepted, it should be merged with dir-spec.txt. Some - preliminaries for authority and caching support should be done during - the 0.1.2.x series; the main deployment should come during the 0.2.0.x - series. - -0.1. Goals and motivation: voting. - - The current directory system relies on clients downloading separate - network status statements from the caches signed by each directory. - Clients download a new statement every 30 minutes or so, choosing to - replace the oldest statement they currently have. - - This creates a partitioning problem: different clients have different - "most recent" networkstatus sources, and different versions of each - (since authorities change their statements often). - - It also creates a scaling problem: most of the downloaded networkstatus - are probably quite similar, and the redundancy grows as we add more - authorities. - - So if we have clients only download a single multiply signed consensus - network status statement, we can: - - Save bandwidth. - - Reduce client partitioning - - Reduce client-side and cache-side storage - - Simplify client-side voting code (by moving voting away from the - client) - - We should try to do this without: - - Assuming that client-side or cache-side clocks are more correct - than we assume now. - - Assuming that authority clocks are perfectly correct. - - Degrading badly if a few authorities die or are offline for a bit. - - We do not have to perform well if: - - No clique of more than half the authorities can agree about who - the authorities are. - -1. The idea. - - Instead of publishing a network status whenever something changes, - each authority instead publishes a fresh network status only once per - "period" (say, 60 minutes). Authorities either upload this network - status (or "vote") to every other authority, or download every other - authority's "vote" (see 3.1 below for discussion on push vs pull). - - After an authority has (or has become convinced that it won't be able to - get) every other authority's vote, it deterministically computes a - consensus networkstatus, and signs it. Authorities download (or are - uploaded; see 3.1) one another's signatures, and form a multiply signed - consensus. This multiply-signed consensus is what caches cache and what - clients download. - - If an authority is down, authorities vote based on what they *can* - download/get uploaded. - - If an authority is "a little" down and only some authorities can reach - it, authorities try to get its info from other authorities. - - If an authority computes the vote wrong, its signature isn't included on - the consensus. - - Clients use a consensus if it is "trusted": signed by more than half the - authorities they recognize. If clients can't find any such consensus, - they use the most recent trusted consensus they have. If they don't - have any trusted consensus, they warn the user and refuse to operate - (and if DirServers is not the default, beg the user to adapt the list - of authorities). - -2. Details. - -2.0. Versioning - - All documents generated here have version "3" given in their - network-status-version entries. - -2.1. Vote specifications - - Votes in v3 are similar to v2 network status documents. We add these - fields to the preamble: - - "vote-status" -- the word "vote". - - "valid-until" -- the time when this authority expects to publish its - next vote. - - "known-flags" -- a space-separated list of flags that will sometimes - be included on "s" lines later in the vote. - - "dir-source" -- as before, except the "hostname" part MUST be the - authority's nickname, which MUST be unique among authorities, and - MUST match the nickname in the "directory-signature" entry. - - Authorities SHOULD cache their most recently generated votes so they - can persist them across restarts. Authorities SHOULD NOT generate - another document until valid-until has passed. - - Router entries in the vote MUST be sorted in ascending order by router - identity digest. The flags in "s" lines MUST appear in alphabetical - order. - - Votes SHOULD be synchronized to half-hour publication intervals (one - hour? XXX say more; be more precise.) - - XXXX some way to request older networkstatus docs? - -2.2. Consensus directory specifications - - Consensuses are like v3 votes, except for the following fields: - - "vote-status" -- the word "consensus". - - "published" is the latest of all the published times on the votes. - - "valid-until" is the earliest of all the valid-until times on the - votes. - - "dir-source" and "fingerprint" and "dir-signing-key" and "contact" - are included for each authority that contributed to the vote. - - "vote-digest" for each authority that contributed to the vote, - calculated as for the digest in the signature on the vote. [XXX - re-English this sentence] - - "client-versions" and "server-versions" are sorted in ascending - order based on version-spec.txt. - - "dir-options" and "known-flags" are not included. -[XXX really? why not list the ones that are used in the consensus? -For example, right now BadExit is in use, but no servers would be -labelled BadExit, and it's still worth knowing that it was considered -by the authorities. -RD] - - The fields MUST occur in the following order: - "network-status-version" - "vote-status" - "published" - "valid-until" - For each authority, sorted in ascending order of nickname, case- - insensitively: - "dir-source", "fingerprint", "contact", "dir-signing-key", - "vote-digest". - "client-versions" - "server-versions" - - The signatures at the end of the document appear as multiple instances - of directory-signature, sorted in ascending order by nickname, - case-insensitively. - - A router entry should be included in the result if it is included by more - than half of the authorities (total authorities, not just those whose votes - we have). A router entry has a flag set if it is included by more than - half of the authorities who care about that flag. [XXXX this creates an - incentive for attackers to DOS authorities whose votes they don't like. - Can we remember what flags people set the last time we saw them? -NM] - [Which 'we' are we talking here? The end-users never learn which - authority sets which flags. So you're thinking the authorities - should record the last vote they saw from each authority and if it's - within a week or so, count all the flags that it advertised as 'no' - votes? Plausible. -RD] - - The signature hash covers from the "network-status-version" line through - the characters "directory-signature" in the first "directory-signature" - line. - - Consensus directories SHOULD be rejected if they are not signed by more - than half of the known authorities. - -2.2.1. Detached signatures - - Assuming full connectivity, every authority should compute and sign the - same consensus directory in each period. Therefore, it isn't necessary to - download the consensus computed by each authority; instead, the authorities - only push/fetch each others' signatures. A "detached signature" document - contains a single "consensus-digest" entry and one or more - directory-signature entries. [XXXX specify more.] - -2.3. URLs and timelines - -2.3.1. URLs and timeline used for agreement - - An authority SHOULD publish its vote immediately at the start of each voting - period. It does this by making it available at - http:///tor/status-vote/current/authority.z - and sending it in an HTTP POST request to each other authority at the URL - http:///tor/post/vote - - If, N minutes after the voting period has begun, an authority does not have - a current statement from another authority, the first authority retrieves - the other's statement. - - Once an authority has a vote from another authority, it makes it available - at - http:///tor/status-vote/current/.z - where is the fingerprint of the other authority's identity key. - - The consensus network status, along with as many signatures as the server - currently knows, should be available at - http:///tor/status-vote/current/consensus.z - All of the detached signatures it knows for consensus status should be - available at: - http:///tor/status-vote/current/consensus-signatures.z - - Once an authority has computed and signed a consensus network status, it - should send its detached signature to each other authority in an HTTP POST - request to the URL: - http:///tor/post/consensus-signature - - - [XXXX Store votes to disk.] - -2.3.2. Serving a consensus directory - - Once the authority is done getting signatures on the consensus directory, - it should serve it from: - http:///tor/status/consensus.z - - Caches SHOULD download consensus directories from an authority and serve - them from the same URL. - -2.3.3. Timeline and synchronization - - [XXXX] - -2.4. Distributing routerdescs between authorities - - Consensus will be more meaningful if authorities take steps to make sure - that they all have the same set of descriptors _before_ the voting - starts. This is safe, since all descriptors are self-certified and - timestamped: it's always okay to replace a signed descriptor with a more - recent one signed by the same identity. - - In the long run, we might want some kind of sophisticated process here. - For now, since authorities already download one another's networkstatus - documents and use them to determine what descriptors to download from one - another, we can rely on this existing mechanism to keep authorities up to - date. - - [We should do a thorough read-through of dir-spec again to make sure - that the authorities converge on which descriptor to "prefer" for - each router. Right now the decision happens at the client, which is - no longer the right place for it. -RD] - -3. Questions and concerns - -3.1. Push or pull? - - The URLs above define a push mechanism for publishing votes and consensus - signatures via HTTP POST requests, and a pull mechanism for downloading - these documents via HTTP GET requests. As specified, every authority will - post to every other. The "download if no copy has been received" mechanism - exists only as a fallback. - -4. Migration - - * It would be cool if caches could get ready to download consensus - status docs, verify enough signatures, and serve them now. That way - once stuff works all we need to do is upgrade the authorities. Caches - don't need to verify the correctness of the format so long as it's - signed (or maybe multisigned?). We need to make sure that caches back - off very quickly from downloading consensus docs until they're - actually implemented. - diff --git a/doc/spec/proposals/102-drop-opt.txt b/doc/spec/proposals/102-drop-opt.txt deleted file mode 100644 index 8f6a38ae6..000000000 --- a/doc/spec/proposals/102-drop-opt.txt +++ /dev/null @@ -1,40 +0,0 @@ -Filename: 102-drop-opt.txt -Title: Dropping "opt" from the directory format -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: Jan 2007 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - This document proposes a change in the format used to transmit router and - directory information. - - This proposal has been accepted, implemented, and merged into dir-spec.txt. - -Proposal: - - The "opt" keyword in Tor's directory formats was originally intended to - mean, "it is okay to ignore this entry if you don't understand it"; the - default behavior has been "discard a routerdesc if it contains entries you - don't recognize." - - But so far, every new flag we have added has been marked 'opt'. It would - probably make sense to change the default behavior to "ignore unrecognized - fields", and add the statement that clients SHOULD ignore fields they don't - recognize. As a meta-principle, we should say that clients and servers - MUST NOT have to understand new fields in order to use directory documents - correctly. - - Of course, this will make it impossible to say, "The format has changed a - lot; discard this quietly if you don't understand it." We could do that by - adding a version field. - -Status: - - * We stopped requiring it as of 0.1.2.5-alpha. We'll stop generating it - once earlier formats are obsolete. - - diff --git a/doc/spec/proposals/103-multilevel-keys.txt b/doc/spec/proposals/103-multilevel-keys.txt deleted file mode 100644 index ef51e1804..000000000 --- a/doc/spec/proposals/103-multilevel-keys.txt +++ /dev/null @@ -1,206 +0,0 @@ -Filename: 103-multilevel-keys.txt -Title: Splitting identity key from regularly used signing key. -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: Jan 2007 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - This document proposes a change in the way identity keys are used, so that - highly sensitive keys can be password-protected and seldom loaded into RAM. - - It presents options; it is not yet a complete proposal. - -Proposal: - - Replacing a directory authority's identity key in the event of a compromise - would be tremendously annoying. We'd need to tell every client to switch - their configuration, or update to a new version with an uploaded list. So - long as some weren't upgraded, they'd be at risk from whoever had - compromised the key. - - With this in mind, it's a shame that our current protocol forces us to - store identity keys unencrypted in RAM. We need some kind of signing key - stored unencrypted, since we need to generate new descriptors/directories - and rotate link and onion keys regularly. (And since, of course, we can't - ask server operators to be on-hand to enter a passphrase every time we - want to rotate keys or sign a descriptor.) - - The obvious solution seems to be to have a signing-only key that lives - indefinitely (months or longer) and signs descriptors and link keys, and a - separate identity key that's used to sign the signing key. Tor servers - could run in one of several modes: - 1. Identity key stored encrypted. You need to pick a passphrase when - you enable this mode, and re-enter this passphrase every time you - rotate the signing key. - 1'. Identity key stored separate. You save your identity key to a - floppy, and use the floppy when you need to rotate the signing key. - 2. All keys stored unencrypted. In this case, we might not want to even - *have* a separate signing key. (We'll need to support no-separate- - signing-key mode anyway to keep old servers working.) - 3. All keys stored encrypted. You need to enter a passphrase to start - Tor. - (Of course, we might not want to implement all of these.) - - Case 1 is probably most usable and secure, if we assume that people don't - forget their passphrases or lose their floppies. We could mitigate this a - bit by encouraging people to PGP-encrypt their passphrases to themselves, - or keep a cleartext copy of their secret key secret-split into a few - pieces, or something like that. - - Migration presents another difficulty, especially with the authorities. If - we use the current set of identity keys as the new identity keys, we're in - the position of having sensitive keys that have been stored on - media-of-dubious-encryption up to now. Also, we need to keep old clients - (who will expect descriptors to be signed by the identity keys they know - and love, and who will not understand signing keys) happy. - -A possible solution: - - One thing to consider is that router identity keys are not very sensitive: - if an OR disappears and reappears with a new key, the network treats it as - though an old router had disappeared and a new one had joined the network. - The Tor network continues unharmed; this isn't a disaster. - - Thus, the ideas above are mostly relevant for authorities. - - The most straightforward solution for the authorities is probably to take - advantage of the protocol transition that will come with proposal 101, and - introduce a new set of signing _and_ identity keys used only to sign votes - and consensus network-status documents. Signing and identity keys could be - delivered to users in a separate, rarely changing "keys" document, so that - the consensus network-status documents wouldn't need to include N signing - keys, N identity keys, and N certifications. - - Note also that there is no reason that the identity/signing keys used by - directory authorities would necessarily have to be the same as the identity - keys those authorities use in their capacity as routers. Decoupling these - keys would give directory authorities the following set of keys: - - Directory authority identity: - Highly confidential; stored encrypted and/or offline. Used to - identity directory authorities. Shipped with clients. Used to - sign Directory authority signing keys. - - Directory authority signing key: - Stored online, accessible to regular Tor process. Used to sign - votes and consensus directories. Downloaded as part of a "keys" - document. - - [Administrators SHOULD rotate their signing keys every month or - two, just to keep in practice and keep from forgetting the - password to the authority identity.] - - V1-V2 directory authority identity: - Stored online, never changed. Used to sign legacy network-status - and directory documents. - - Router identity: - Stored online, seldom changed. Used to sign server descriptors - for this authority in its role as a router. Implicitly certified - by being listed in network-status documents. - - Onion key, link key: - As in tor-spec.txt - - -Extensions to Proposal 101. - - Define a new document type, "Key certificate". It contains the - following fields, in order: - - "dir-key-certificate-version": As network-status-version. Must be - "3". - "fingerprint": Hex fingerprint, with spaces, based on the directory - authority's identity key. - "dir-identity-key": The long-term identity key for this authority. - "dir-key-published": The time when this directory's signing key was - last changed. - "dir-key-expires": A time after which this key is no longer valid. - "dir-signing-key": As in proposal 101. - "dir-key-certification": A signature of the above fields, in order. - The signed material extends from the beginning of - "dir-key-certicate-version" through the newline after - "dir-key-certification". The identity key is used to generate - this signature. - - These elements together constitute a "key certificate". These are - generated offline when starting a v3 authority. Private identity - keys SHOULD be stored offline, encrypted, or both. A running - authority only needs access to the signing key. - - Unlike other keys currently used by Tor, the authority identity - keys and directory signing keys MAY be longer than 1024 bits. - (They SHOULD be 2048 bits or longer; they MUST NOT be shorter than - 1024.) - - Vote documents change as follows: - - A key certificate MUST be included in-line in every vote document. With - the exception of "fingerprint", its elements MUST NOT appear in consensus - documents. - - Consensus network statuses change as follows: - - Remove dir-signing-key. - - Change "directory-signature" to take a fingerprint of the authority's - identity key and a fingerprint of the authority's current signing key - rather than the authority's nickname. - - Change "dir-source" to take the a fingerprint of the authority's - identity key rather than the authority's nickname or hostname. - - Add a new document type: - - A "keys" document contains all currently known key certificates. - All authorities serve it at - - http:///tor/status/keys.z - - Caches and clients download the keys document whenever they receive a - consensus vote that uses a key they do not recognize. Caches download - from authorities; clients download from caches. - - Processing votes: - - When receiving a vote, authorities check to see if the key - certificate for the voter is different from the one they have. If - the key certificate _is_ different, and its dir-key-published is - more recent than the most recently known one, and it is - well-formed and correctly signed with the correct identity key, - then authorities remember it as the new canonical key certificate - for that voter. - - A key certificate is invalid if any of the following hold: - * The version is unrecognized. - * The fingerprint does not match the identity key. - * The identity key or the signing key is ill-formed. - * The published date is very far in the past or future. - - * The signature is not a valid signature of the key certificate - generated with the identity key. - - When processing the signatures on consensus, clients and caches act as - follows: - - 1. Only consider the directory-signature entries whose identity - key hashes match trusted authorities. - - 2. If any such entries have signing key hashes that match unknown - signing keys, download a new keys document. - - 3. For every entry with a known (identity key,signing key) pair, - check the signature on the document. - - 4. If the document has been signed by more than half of the - authorities the client recognizes, treat the consensus as - correctly signed. - - If not, but the number entries with known identity keys but - unknown signing keys might be enough to make the consensus - correctly signed, do not use the consensus, but do not discard - it until we have a new keys document. diff --git a/doc/spec/proposals/104-short-descriptors.txt b/doc/spec/proposals/104-short-descriptors.txt deleted file mode 100644 index a1c42c8ff..000000000 --- a/doc/spec/proposals/104-short-descriptors.txt +++ /dev/null @@ -1,183 +0,0 @@ -Filename: 104-short-descriptors.txt -Title: Long and Short Router Descriptors -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: Jan 2007 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - This document proposes moving unused-by-clients information from regular - router descriptors into a new "extra info" router descriptor. - -Proposal: - - Some of the costliest fields in the current directory protocol are ones - that no client actually uses. In particular, the "read-history" and - "write-history" fields are used only by the authorities for monitoring the - status of the network. If we took them out, the size of a compressed list - of all the routers would fall by about 60%. (No other disposable field - would save much more than 2%.) - - We propose to remove these fields from descriptors, and and have them - uploaded as a part of a separate signed "extra info" to the authorities. - This document will be signed. A hash of this document will be included in - the regular descriptors. - - (We considered another design, where routers would generate and upload a - short-form and a long-form descriptor. Only the short-form descriptor would - ever be used by anybody for routing. The long-form descriptor would be - used only for analytics and other tools. We decided against this because - well-behaved tools would need to download short-form descriptors too (as - these would be the only ones indexed), and hence get redundant info. Badly - behaved tools would download only long-form descriptors, and expose - themselves to partitioning attacks.) - -Other disposable fields: - - Clients don't need these fields, but removing them doesn't help bandwidth - enough to be worthwhile. - contact (save about 1%) - fingerprint (save about 3%) - - We could represent these fields more succinctly, but removing them would - only save 1%. (!) - reject - accept - (Apparently, exit polices are highly compressible.) - - [Does size-on-disk matter to anybody? Some clients and servers don't - have much disk, or have really slow disk (e.g. USB). And we don't - store caches compressed right now. -RD] - -Specification: - - 1. Extra Info Format. - - An "extra info" descriptor contains the following fields: - - "extra-info" Nickname Fingerprint - Identifies what router this is an extra info descriptor for. - Fingerprint is encoded in hex (using upper-case letters), with - no spaces. - - "published" As currently documented in dir-spec.txt. It MUST match the - "published" field of the descriptor published at the same time. - - "read-history" - "write-history" - As currently documented in dir-spec.txt. Optional. - - "router-signature" NL Signature NL - - A signature of the PKCS1-padded hash of the entire extra info - document, taken from the beginning of the "extra-info" line, through - the newline after the "router-signature" line. An extra info - document is not valid unless the signature is performed with the - identity key whose digest matches FINGERPRINT. - - The "extra-info" field is required and MUST appear first. The - router-signature field is required and MUST appear last. All others are - optional. As for other documents, unrecognized fields must be ignored. - - 2. Existing formats - - Implementations that use "read-history" and "write-history" SHOULD - continue accepting router descriptors that contain them. (Prior to - 0.2.0.x, this information was encoded in ordinary router descriptors; - in any case they have always been listed as opt, so they should be - accepted anyway.) - - Add these fields to router descriptors: - - "extra-info-digest" Digest - "Digest" is a hex-encoded digest (using upper-case characters) - of the router's extra-info document, as signed in the router's - extra-info. (If this field is absent, no extra-info-digest - exists.) - - "caches-extra-info" - Present if this router is a directory cache that provides - extra-info documents, or an authority that handles extra-info - documents. - - (Since implementations before 0.1.2.5-alpha required that the "opt" - keyword precede any unrecognized entry, these keys MUST be preceded - with "opt" until 0.1.2.5-alpha is obsolete.) - - 3. New communications rules - - Servers SHOULD generate and upload one extra-info document after each - descriptor they generate and upload; no more, no less. Servers MUST - upload the new descriptor before they upload the new extra-info. - - Authorities receiving an extra-info document SHOULD verify all of the - following: - * They have a router descriptor for some server with a matching - nickname and identity fingerprint. - * That server's identity key has been used to sign the extra-info - document. - * The extra-info-digest field in the router descriptor matches - the digest of the extra-info document. - * The published fields in the two documents match. - - Authorities SHOULD drop extra-info documents that do not meet these - criteria. - - Extra-info documents MAY be uploaded as part of the same HTTP post as - the router descriptor, or separately. Authorities MUST accept both - methods. - - Authorities SHOULD try to fetch extra-info documents from one another if - they do not have one matching the digest declared in a router - descriptor. - - Caches that are running locally with a tool that needs to use extra-info - documents MAY download and store extra-info documents. They should do - so when they notice that the recommended descriptor has an - extra-info-digest not matching any extra-info document they currently - have. (Caches not running on a host that needs to use extra-info - documents SHOULD NOT download or cache them.) - - 4. New URLs - - http:///tor/extra/d/... - http:///tor/extra/fp/... - http:///tor/extra/all[.z] - (As for /tor/server/ URLs: supports fetching extra-info documents - by their digest, by the fingerprint of their servers, or all - at once. When serving by fingerprint, we serve the extra-info - that corresponds to the descriptor we would serve by that - fingerprint. Only directory authorities are guaranteed to support - these URLs.) - - http:///tor/extra/authority[.z] - (The extra-info document for this router.) - - Extra-info documents are uploaded to the same URLs as regular - router descriptors. - -Migration: - - For extra info approach: - * First: - * Authorities should accept extra info, and support serving it. - * Routers should upload extra info once authorities accept it. - * Caches should support an option to download and cache it, once - authorities serve it. - * Tools should be updated to use locally cached information. - These tools include: - lefkada's exit.py script. - tor26's noreply script and general directory cache. - https://nighteffect.us/tns/ for its graphs - and check with or-talk for the rest, once it's time. - - * Set a cutoff time for including bandwidth in router descriptors, so - that tools that use bandwidth info know that they will need to fetch - extra info documents. - - * Once tools that want bandwidth info support fetching extra info: - * Have routers stop including bandwidth info in their router - descriptors. diff --git a/doc/spec/proposals/105-handshake-revision.txt b/doc/spec/proposals/105-handshake-revision.txt deleted file mode 100644 index f6c209e71..000000000 --- a/doc/spec/proposals/105-handshake-revision.txt +++ /dev/null @@ -1,325 +0,0 @@ -Filename: 105-handshake-revision.txt -Title: Version negotiation for the Tor protocol. -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson, Roger Dingledine -Created: Jan 2007 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - This document was extracted from a modified version of tor-spec.txt that we - had written before the proposal system went into place. It adds two new - cells types to the Tor link connection setup handshake: one used for - version negotiation, and another to prevent MITM attacks. - - This proposal is partially implemented, and partially proceded by - proposal 130. - -Motivation: Tor versions - - Our *current* approach to versioning the Tor protocol(s) has been as - follows: - - All changes must be backward compatible. - - It's okay to add new cell types, if they would be ignored by previous - versions of Tor. - - It's okay to add new data elements to cells, if they would be - ignored by previous versions of Tor. - - For forward compatibility, Tor must ignore cell types it doesn't - recognize, and ignore data in those cells it doesn't expect. - - Clients can inspect the version of Tor declared in the platform line - of a router's descriptor, and use that to learn whether a server - supports a given feature. Servers, however, aren't assumed to all - know about each other, and so don't know the version of who they're - talking to. - - This system has these problems: - - It's very hard to change fundamental aspects of the protocol, like the - cell format, the link protocol, any of the various encryption schemes, - and so on. - - The router-to-router link protocol has remained more-or-less frozen - for a long time, since we can't easily have an OR use new features - unless it knows the other OR will understand them. - - We need to resolve these problems because: - - Our cipher suite is showing its age: SHA1/AES128/RSA1024/DH1024 will - not seem like the best idea for all time. - - There are many ideas circulating for multiple cell sizes; while it's - not obvious whether these are safe, we can't do them at all without a - mechanism to permit them. - - There are many ideas circulating for alternative circuit building and - cell relay rules: they don't work unless they can coexist in the - current network. - - If our protocol changes a lot, it's hard to describe any coherent - version of it: we need to say "the version that Tor versions W through - X use when talking to versions Y through Z". This makes analysis - harder. - -Motivation: Preventing MITM attacks - - TLS prevents a man-in-the-middle attacker from reading or changing the - contents of a communication. It does not, however, prevent such an - attacker from observing timing information. Since timing attacks are some - of the most effective against low-latency anonymity nets like Tor, we - should take more care to make sure that we're not only talking to who - we think we're talking to, but that we're using the network path we - believe we're using. - -Motivation: Signed clock information - - It's very useful for Tor instances to know how skewed they are relative - to one another. The only way to find out currently has been to download - directory information, and check the Date header--but this is not - authenticated, and hence subject to modification on the wire. Using - BEGIN_DIR to create an authenticated directory stream through an existing - circuit is better, but that's an extra step and it might be nicer to - learn the information in the course of the regular protocol. - -Proposal: - -1.0. Version numbers - - The node-to-node TLS-based "OR connection" protocol and the multi-hop - "circuit" protocol are versioned quasi-independently. - - Of course, some dependencies will continue to exist: Certain versions - of the circuit protocol may require a minimum version of the connection - protocol to be used. The connection protocol affects: - - Initial connection setup, link encryption, transport guarantees, - etc. - - The allowable set of cell commands - - Allowable formats for cells. - - The circuit protocol determines: - - How circuits are established and maintained - - How cells are decrypted and relayed - - How streams are established and maintained. - - Version numbers are incremented for backward-incompatible protocol changes - only. Backward-compatible changes are generally implemented by adding - additional fields to existing structures; implementations MUST ignore - fields they do not expect. Unused portions of cells MUST be set to zero. - - Though versioning the protocol will make it easier to maintain backward - compatibility with older versions of Tor, we will nevertheless continue to - periodically drop support for older protocols, - - to keep the implementation from growing without bound, - - to limit the maintenance burden of patching bugs in obsolete Tors, - - to limit the testing burden of verifying that many old protocol - versions continue to be implemented properly, and - - to limit the exposure of the network to protocol versions that are - expensive to support. - - The Tor protocol as implemented through the 0.1.2.x Tor series will be - called "version 1" in its link protocol and "version 1" in its relay - protocol. Versions of the Tor protocol so old as to be incompatible with - Tor 0.1.2.x can be considered to be version 0 of each, and are not - supported. - -2.1. VERSIONS cells - - When a Tor connection is established, both parties normally send a - VERSIONS cell before sending any other cells. (But see below.) - - VersionsLen [2 byte] - Versions [VersionsLen bytes] - - "Versions" is a sequence of VersionsLen bytes. Each value between 1 and - 127 inclusive represents a single version; current implementations MUST - ignore other bytes. Parties should list all of the versions which they - are able and willing to support. Parties can only communicate if they - have some connection protocol version in common. - - Version 0.2.0.x-alpha and earlier don't understand VERSIONS cells, - and therefore don't support version negotiation. Thus, waiting until - the other side has sent a VERSIONS cell won't work for these servers: - if the other side sends no cells back, it is impossible to tell - whether they - have sent a VERSIONS cell that has been stalled, or whether they have - dropped our own VERSIONS cell as unrecognized. Therefore, we'll - change the TLS negotiation parameters so that old parties can still - negotiate, but new parties can recognize each other. Immediately - after a TLS connection has been established, the parties check - whether the other side negotiated the connection in an "old" way or a - "new" way. If either party negotiated in the "old" way, we assume a - v1 connection. Otherwise, both parties send VERSIONS cells listing - all their supported versions. Upon receiving the other party's - VERSIONS cell, the implementation begins using the highest-valued - version common to both cells. If the first cell from the other party - has a recognized command, and is _not_ a VERSIONS cell, we assume a - v1 protocol. - - (For more detail on the TLS protocol change, see forthcoming draft - proposals from Steven Murdoch.) - - Implementations MUST discard VERSIONS cells that are not the first - recognized cells sent on a connection. - - The VERSIONS cell must be sent as a v1 cell (2 bytes of circuitID, 1 - byte of command, 509 bytes of payload). - - [NOTE: The VERSIONS cell is assigned the command number 7.] - -2.2. MITM-prevention and time checking - - If we negotiate a v2 connection or higher, the second cell we send SHOULD - be a NETINFO cell. Implementations SHOULD NOT send NETINFO cells at other - times. - - A NETINFO cell contains: - Timestamp [4 bytes] - Other OR's address [variable] - Number of addresses [1 byte] - This OR's addresses [variable] - - Timestamp is the OR's current Unix time, in seconds since the epoch. If - an implementation receives time values from many ORs that - indicate that its clock is skewed, it SHOULD try to warn the - administrator. (We leave the definition of 'many' intentionally vague - for now.) - - Before believing the timestamp in a NETINFO cell, implementations - SHOULD compare the time at which they received the cell to the time - when they sent their VERSIONS cell. If the difference is very large, - it is likely that the cell was delayed long enough that its - contents are out of date. - - Each address contains Type/Length/Value as used in Section 6.4 of - tor-spec.txt. The first address is the one that the party sending - the NETINFO cell believes the other has -- it can be used to learn - what your IP address is if you have no other hints. - The rest of the addresses are the advertised addresses of the party - sending the NETINFO cell -- we include them - to block a man-in-the-middle attack on TLS that lets an attacker bounce - traffic through his own computers to enable timing and packet-counting - attacks. - - A Tor instance should use the other Tor's reported address - information as part of logic to decide whether to treat a given - connection as suitable for extending circuits to a given address/ID - combination. When we get an extend request, we use an - existing OR connection if the ID matches, and ANY of the following - conditions hold: - - The IP matches the requested IP. - - We know that the IP we're using is canonical because it was - listed in the NETINFO cell. - - We know that the IP we're using is canonical because it was - listed in the server descriptor. - - [NOTE: The NETINFO cell is assigned the command number 8.] - -Discussion: Versions versus feature lists - - Many protocols negotiate lists of available features instead of (or in - addition to) protocol versions. While it's possible that some amount of - feature negotiation could be supported in a later Tor, we should prefer to - use protocol versions whenever possible, for reasons discussed in - the "Anonymity Loves Company" paper. - -Discussion: Bytes per version, versions per cell - - This document provides for a one-byte count of how many versions a Tor - supports, and allows one byte per version. Thus, it can only support only - 254 more versions of the protocol beyond the unallocated v0 and the - current v1. If we ever need to split the protocol into 255 incompatible - versions, we've probably screwed up badly somewhere. - - Nevertheless, here are two ways we could support more versions: - - Change the version count to a two-byte field that counts the number of - _bytes_ used, and use a UTF8-style encoding: versions 0 through 127 - take one byte to encode, versions 128 through 2047 take two bytes to - encode, and so on. We wouldn't need to parse any version higher than - 127 right now, since all bytes used to encode higher versions would - have their high bit set. - - We'd still have a limit of 380 simultaneously versions that could be - declared in any version. This is probably okay. - - - Decide that if we need to support more versions, we can add a - MOREVERSIONS cell that gets sent before the VERSIONS cell. The spec - above requires Tors to ignore unrecognized cell types that they get - before the first VERSIONS cell, and still allows version negotiation - to - succeed. - - [Resolution: Reserve the high bit and the v0 value for later use. If - we ever have more live versions than we can fit in a cell, we've made a - bad design decision somewhere along the line.] - -Discussion: Reducing round-trips - - It might be appealing to see if we can cram more information in the - initial VERSIONS cell. For example, the contents of NETINFO will pretty - soon be sent by everybody before any more information is exchanged, but - decoupling them from the version exchange increases round-trips. - - Instead, we could speculatively include handshaking information at - the end of a VERSIONS cell, wrapped in a marker to indicate, "if we wind - up speaking VERSION 2, here's the NETINFO I'll send. Otherwise, ignore - this." This could be extended to opportunistically reduce round trips - when possible for future versions when we guess the versions right. - - Of course, we'd need to be careful about using a feature like this: - - We don't want to include things that are expensive to compute, - like PK signatures or proof-of-work. - - We don't want to speculate as a mobile client: it may leak our - experience with the server in question. - -Discussion: Advertising versions in routerdescs and networkstatuses. - - In network-statuses: - - The networkstatus "v" line now has the format: - "v" IMPLEMENTATION IMPL-VERSION "Link" LINK-VERSION-LIST - "Circuit" CIRCUIT-VERSION-LIST NL - - LINK-VERSION-LIST and CIRCUIT-VERSION-LIST are comma-separated lists of - supported version numbers. IMPLEMENTATION is the name of the - implementation of the Tor protocol (e.g., "Tor"), and IMPL-VERSION is the - version of the implementation. - - Examples: - v Tor 0.2.5.1-alpha Link 1,2,3 Circuit 2,5 - - v OtherOR 2000+ Link 3 Circuit 5 - - Implementations that release independently of the Tor codebase SHOULD NOT - use "Tor" as the value of their IMPLEMENTATION. - - Additional fields on the "v" line MUST be ignored. - - In router descriptors: - - The router descriptor should contain a line of the form, - "protocols" "Link" LINK-VERSION-LIST "Circuit" CIRCUIT_VERSION_LIST - - Additional fields on the "protocols" line MUST be ignored. - - [Versions of Tor before 0.1.2.5-alpha rejected router descriptors with - unrecognized items; the protocols line should be preceded with an "opt" - until these Tors are obsolete.] - -Security issues: - - Client partitioning is the big danger when we introduce new versions; if a - client supports some very unusual set of protocol versions, it will stand - out from others no matter where it goes. If a server supports an unusual - version, it will get a disproportionate amount of traffic from clients who - prefer that version. We can mitigate this somewhat as follows: - - - Do not have clients prefer any protocol version by default until that - version is widespread. (First introduce the new version to servers, - and have clients admit to using it only when configured to do so for - testing. Then, once many servers are running the new protocol - version, enable its use by default.) - - - Do not multiply protocol versions needlessly. - - - Encourage protocol implementors to implement the same protocol version - sets as some popular version of Tor. - - - Disrecommend very old/unpopular versions of Tor via the directory - authorities' RecommmendedVersions mechanism, even if it is still - technically possible to use them. - diff --git a/doc/spec/proposals/106-less-tls-constraint.txt b/doc/spec/proposals/106-less-tls-constraint.txt deleted file mode 100644 index 35d6bf106..000000000 --- a/doc/spec/proposals/106-less-tls-constraint.txt +++ /dev/null @@ -1,113 +0,0 @@ -Filename: 106-less-tls-constraint.txt -Title: Checking fewer things during TLS handshakes -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 9-Feb-2007 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - This document proposes that we relax our requirements on the context of - X.509 certificates during initial TLS handshakes. - -Motivation: - - Later, we want to try harder to avoid protocol fingerprinting attacks. - This means that we'll need to make our connection handshake look closer - to a regular HTTPS connection: one certificate on the server side and - zero certificates on the client side. For now, about the best we - can do is to stop requiring things during handshake that we don't - actually use. - -What we check now, and where we check it: - - tor_tls_check_lifetime: - peer has certificate - notBefore <= now <= notAfter - - tor_tls_verify: - peer has at least one certificate - There is at least one certificate in the chain - At least one of the certificates in the chain is not the one used to - negotiate the connection. (The "identity cert".) - The certificate _not_ used to negotiate the connection has signed the - link cert - - tor_tls_get_peer_cert_nickname: - peer has a certificate. - certificate has a subjectName. - subjectName has a commonName. - commonName consists only of characters in LEGAL_NICKNAME_CHARACTERS. [2] - - tor_tls_peer_has_cert: - peer has a certificate. - - connection_or_check_valid_handshake: - tor_tls_peer_has_cert [1] - tor_tls_get_peer_cert_nickname [1] - tor_tls_verify [1] - If nickname in cert is a known, named router, then its identity digest - must be as expected. - If we initiated the connection, then we got the identity digest we - expected. - - USEFUL THINGS WE COULD DO: - - [1] We could just not force clients to have any certificate at all, let alone - an identity certificate. Internally to the code, we could assign the - identity_digest field of these or_connections to a random number, or even - not add them to the identity_digest->or_conn map. - [so if somebody connects with no certs, we let them. and mark them as - a client and don't treat them as a server. great. -rd] - - [2] Instead of using a restricted nickname character set that makes our - commonName structure look unlike typical SSL certificates, we could treat - the nickname as extending from the start of the commonName up to but not - including the first non-nickname character. - - Alternatively, we could stop checking commonNames entirely. We don't - actually _do_ anything based on the nickname in the certificate, so - there's really no harm in letting every router have any commonName it - wants. - [this is the better choice -rd] - [agreed. -nm] - -REMAINING WAYS TO RECOGNIZE CLIENT->SERVER CONNECTIONS: - - Assuming that we removed the above requirements, we could then (in a later - release) have clients not send certificates, and sometimes and started - making our DNs a little less formulaic, client->server OR connections would - still be recognizable by: - having a two-certificate chain sent by the server - using a particular set of ciphersuites - traffic patterns - probing the server later - -OTHER IMPLICATIONS: - - If we stop verifying the above requirements: - - It will be slightly (but only slightly) more common to connect to a non-Tor - server running TLS, and believe that you're talking to a Tor server (until - you send the first cell). - - It will be far easier for non-Tor SSL clients to accidentally connect to - Tor servers and speak HTTPS or whatever to them. - - If, in a later release, we have clients not send certificates, and we make - DNs less recognizable: - - If clients don't send certs, servers don't need to verify them: win! - - If we remove these restrictions, it will be easier for people to write - clients to fuzz our protocol: sorta win! - - If clients don't send certs, they look slightly less like servers. - -OTHER SPEC CHANGES: - - When a client doesn't give us an identity, we should never extend any - circuits to it (duh), and we should allow it to set circuit ID however it - wants. diff --git a/doc/spec/proposals/107-uptime-sanity-checking.txt b/doc/spec/proposals/107-uptime-sanity-checking.txt deleted file mode 100644 index b11be8938..000000000 --- a/doc/spec/proposals/107-uptime-sanity-checking.txt +++ /dev/null @@ -1,56 +0,0 @@ -Filename: 107-uptime-sanity-checking.txt -Title: Uptime Sanity Checking -Version: $Revision$ -Last-Modified: $Date$ -Author: Kevin Bauer & Damon McCoy -Created: 8-March-2007 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - This document describes how to cap the uptime that is used when computing - which routers are marked as stable such that highly stable routers cannot - be displaced by malicious routers that report extremely high uptime - values. - - This is similar to how bandwidth is capped at 1.5MB/s. - -Motivation: - - It has been pointed out that an attacker can displace all stable nodes and - entry guard nodes by reporting high uptimes. This is an easy fix that will - prevent highly stable nodes from being displaced. - -Security implications: - - It should decrease the effectiveness of routing attacks that report high - uptimes while not impacting the normal routing algorithms. - -Specification: - - So we could patch Section 3.1 of dir-spec.txt to say: - - "Stable" -- A router is 'Stable' if it is running, valid, not - hibernating, and either its uptime is at least the median uptime for - known running, valid, non-hibernating routers, or its uptime is at - least 30 days. Routers are never called stable if they are running - a version of Tor known to drop circuits stupidly. (0.1.1.10-alpha - through 0.1.1.16-rc are stupid this way.) - -Compatibility: - - There should be no compatibility issues due to uptime capping. - -Implementation: - - Implemented and merged into dir-spec in 0.2.0.0-alpha-dev (r9788). - -Discussion: - - Initially, this proposal set the maximum at 60 days, not 30; the 30 day - limit and spec wording was suggested by Roger in an or-dev post on 9 March - 2007. - - This proposal also led to 108-mtbf-based-stability.txt - diff --git a/doc/spec/proposals/108-mtbf-based-stability.txt b/doc/spec/proposals/108-mtbf-based-stability.txt deleted file mode 100644 index 2c6648153..000000000 --- a/doc/spec/proposals/108-mtbf-based-stability.txt +++ /dev/null @@ -1,90 +0,0 @@ -Filename: 108-mtbf-based-stability.txt -Title: Base "Stable" Flag on Mean Time Between Failures -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 10-Mar-2007 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - This document proposes that we change how directory authorities set the - stability flag from inspection of a router's declared Uptime to the - authorities' perceived mean time between failure for the router. - -Motivation: - - Clients prefer nodes that the authorities call Stable. This flag is (as - of 0.2.0.0-alpha-dev) set entirely based on the node's declared value for - uptime. This creates an opportunity for malicious nodes to declare - falsely high uptimes in order to get more traffic. - -Spec changes: - - Replace the current rule for setting the Stable flag with: - - "Stable" -- A router is 'Stable' if it is active and its observed Stability - for the past month is at or above the median Stability for active routers. - Routers are never called stable if they are running a version of Tor - known to drop circuits stupidly. (0.1.1.10-alpha through 0.1.1.16-rc - are stupid this way.) - - Stability shall be defined as the weighted mean length of the runs - observed by a given directory authority. A run begins when an authority - decides that the server is Running, and ends when the authority decides - that the server is not Running. In-progress runs are counted when - measuring Stability. When calculating the mean, runs are weighted by - $\alpha ^ t$, where $t$ is time elapsed since the end of the run, and - $0 < \alpha < 1$. Time when an authority is down do not count to the - length of the run. - -Rejected Alternative: - - "A router's Stability shall be defined as the sum of $\alpha ^ d$ for every - $d$ such that the router was considered reachable for the entire day - $d$ days ago. - - This allows a simpler implementation: every day, we multiply - yesterday's Stability by alpha, and if the router was observed to be - available every time we looked today, we add 1. - - Instead of "day", we could pick an arbitrary time unit. We should - pick alpha to be high enough that long-term stability counts, but low - enough that the distant past is eventually forgotten. Something - between .8 and .95 seems right. - - (By requiring that routers be up for an entire day to get their - stability increased, instead of counting fractions of a day, we - capture the notion that stability is more like "probability of - staying up for the next hour" than it is like "probability of being - up at some randomly chosen time over the next hour." The former - notion of stability is far more relevant for long-lived circuits.) - -Limitations: - - Authorities can have false positives and false negatives when trying to - tell whether a router is up or down. So long as these aren't terribly - wrong, and so long as they aren't significantly biased, we should be able - to use them to estimate stability pretty well. - - Probing approaches like the above could miss short incidents of - downtime. If we use the router's declared uptime, we could detect - these: but doing so would penalize routers who reported their uptime - accurately. - -Implementation: - - For now, the easiest way to store this information at authorities - would probably be in some kind of periodically flushed flat file. - Later, we could move to Berkeley db or something if we really had to. - - For each router, an authority will need to store: - The router ID. - Whether the router is up. - The time when the current run started, if the router is up. - The weighted sum length of all previous runs. - The time at which the weighted sum length was last weighted down. - - Servers should probe at random intervals to test whether servers are - running. diff --git a/doc/spec/proposals/109-no-sharing-ips.txt b/doc/spec/proposals/109-no-sharing-ips.txt deleted file mode 100644 index 1a88b00c0..000000000 --- a/doc/spec/proposals/109-no-sharing-ips.txt +++ /dev/null @@ -1,92 +0,0 @@ -Filename: 109-no-sharing-ips.txt -Title: No more than one server per IP address. -Version: $Revision$ -Last-Modified: $Date$ -Author: Kevin Bauer & Damon McCoy -Created: 9-March-2007 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - This document describes a solution to a Sybil attack vulnerability in the - directory servers. Currently, it is possible for a single IP address to - host an arbitrarily high number of Tor routers. We propose that the - directory servers limit the number of Tor routers that may be registered at - a particular IP address to some small (fixed) number, perhaps just one Tor - router per IP address. - - While Tor never uses more than one server from a given /16 in the same - circuit, an attacker with multiple servers in the same place is still - dangerous because he can get around the per-server bandwidth cap that is - designed to prevent a single server from attracting too much of the overall - traffic. - -Motivation: - Since it is possible for an attacker to register an arbitrarily large - number of Tor routers, it is possible for malicious parties to do this - as part of a traffic analysis attack. - -Security implications: - This countermeasure will increase the number of IP addresses that an - attacker must control in order to carry out traffic analysis. - -Specification: - - For each IP address, each directory authority tracks the number of routers - using that IP address, along with their total observed bandwidth. If there - are more than MAX_SERVERS_PER_IP servers at some IP, the authority should - "disable" all but MAX_SERVERS_PER_IP servers. When choosing which servers - to disable, the authority should first disable non-Running servers in - increasing order of observed bandwidth, and then should disable Running - servers in increasing order of bandwidth. - - [[ We don't actually do this part here. -NM - - If the total observed - bandwidth of the remaining non-"disabled" servers exceeds MAX_BW_PER_IP, - the authority should "disable" some of the remaining servers until only one - server remains, or until the remaining observed bandwidth of non-"disabled" - servers is under MAX_BW_PER_IP. - ]] - - Servers that are "disabled" MUST be marked as non-Valid and non-Running. - - MAX_SERVERS_PER_IP is 3. - - MAX_BW_PER_IP is 8 MB per s. - -Compatibility: - - Upon inspection of a directory server, we found that the following IP - addresses have more than one Tor router: - - Scruples 68.5.113.81 ip68-5-113-81.oc.oc.cox.net 443 - WiseUp 68.5.113.81 ip68-5-113-81.oc.oc.cox.net 9001 - Unnamed 62.1.196.71 pc01-megabyte-net-arkadiou.megabyte.gr 9001 - Unnamed 62.1.196.71 pc01-megabyte-net-arkadiou.megabyte.gr 9001 - Unnamed 62.1.196.71 pc01-megabyte-net-arkadiou.megabyte.gr 9001 - aurel 85.180.62.138 e180062138.adsl.alicedsl.de 9001 - sokrates 85.180.62.138 e180062138.adsl.alicedsl.de 9001 - moria1 18.244.0.188 moria.mit.edu 9001 - peacetime 18.244.0.188 moria.mit.edu 9100 - - There may exist compatibility issues with this proposed fix. Reasons why - more than one server would share an IP address include: - - * Testing. moria1, moria2, peacetime, and other morias all run on one - computer at MIT, because that way we get testing. Moria1 and moria2 are - run by Roger, and peacetime is run by Nick. - * NAT. If there are several servers but they port-forward through the same - IP address, ... we can hope that the operators coordinate with each - other. Also, we should recognize that while they help the network in - terms of increased capacity, they don't help as much as they could in - terms of location diversity. But our approach so far has been to take - what we can get. - * People who have more than 1.5MB/s and want to help out more. For - example, for a while Tonga was offering 10MB/s and its Tor server - would only make use of a bit of it. So Roger suggested that he run - two Tor servers, to use more. - -[Note Roger's tweak to this behavior, in -http://archives.seul.org/or/cvs/Oct-2007/msg00118.html] - diff --git a/doc/spec/proposals/110-avoid-infinite-circuits.txt b/doc/spec/proposals/110-avoid-infinite-circuits.txt deleted file mode 100644 index 1834cd34a..000000000 --- a/doc/spec/proposals/110-avoid-infinite-circuits.txt +++ /dev/null @@ -1,122 +0,0 @@ -Filename: 110-avoid-infinite-circuits.txt -Title: Avoiding infinite length circuits -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 13-Mar-2007 -Status: Accepted -Target: 0.2.1.x -Implemented-In: 0.2.1.3-alpha - -History: - - Revised 28 July 2008 by nickm: set K. - Revised 3 July 2008 by nickm: rename from relay_extend to - relay_early. Revise to current migration plan. Allow K cells - over circuit lifetime, not just at start. - -Overview: - - Right now, an attacker can add load to the Tor network by extending a - circuit an arbitrary number of times. Every cell that goes down the - circuit then adds N times that amount of load in overall bandwidth - use. This vulnerability arises because servers don't know their position - on the path, so they can't tell how many nodes there are before them - on the path. - - We propose a new set of relay cells that are distinguishable by - intermediate hops as permitting extend cells. This approach will allow - us to put an upper bound on circuit length relative to the number of - colluding adversary nodes; but there are some downsides too. - -Motivation: - - The above attack can be used to generally increase load all across the - network, or it can be used to target specific servers: by building a - circuit back and forth between two victim servers, even a low-bandwidth - attacker can soak up all the bandwidth offered by the fastest Tor - servers. - - The general attacks could be used as a demonstration that Tor isn't - perfect (leading to yet more media articles about "breaking" Tor), and - the targetted attacks will come into play once we have a reputation - system -- it will be trivial to DoS a server so it can't pass its - reputation checks, in turn impacting security. - -Design: - - We should split RELAY cells into two types: RELAY and RELAY_EARLY. - - Only K (say, 10) Relay_early cells can be sent across a circuit, and - only relay_early cells are allowed to contain extend requests. We - still support obscuring the length of the circuit (if more research - shows us what to do), because Alice can choose how many of the K to - mark as relay_early. Note that relay_early cells *can* contain any - sort of data cell; so in effect it's actually the relay type cells - that are restricted. By default, she would just send the first K - data cells over the stream as relay_early cells, regardless of their - actual type. - - (Note that a circuit that is out of relay_early cells MUST NOT be - cannibalized later, since it can't extend. Note also that it's always okay - to use regular RELAY cells when sending non-EXTEND commands targetted at - the first hop of a circuit, since there is no intermediate hop to try to - learn the relay command type.) - - Each intermediate server would pass on the same type of cell that it - received (either relay or relay_early), and the cell's destination - will be able to learn whether it's allowed to contain an Extend request. - - If an intermediate server receives more than K relay_early cells, or - if it sees a relay cell that contains an extend request, then it - tears down the circuit (protocol violation). - -Security implications: - - The upside is that this limits the bandwidth amplification factor to - K: for an individual circuit to become arbitrary-length, the attacker - would need an adversary-controlled node every K hops, and at that - point the attack is no worse than if the attacker creates N/K separate - K-hop circuits. - - On the other hand, we want to pick a large enough value of K that we - don't mind the cap. - - If we ever want to take steps to hide the number of hops in the circuit - or a node's position in the circuit, this design probably makes that - more complex. - -Migration: - - In 0.2.0, servers speaking v2 or later of the link protocol accept - RELAY_EARLY cells, and pass them on. If the next OR in the circuit - is not speaking the v2 link protocol, the server relays the cell as - a RELAY cell. - - In 0.2.1.3-alpha, clients begin using RELAY_EARLY cells on v2 - connections. This functionality can be safely backported to - 0.2.0.x. Clients should pick a random number betweeen (say) K and - K-2 to send. - - In 0.2.1.3-alpha, servers close any circuit in which more than K - relay_early cells are sent. - - Once all versions the do not send RELAY_EARLY cells are obsolete, - servers can begin to reject any EXTEND requests not sent in a - RELAY_EARLY cell. - -Parameters: - - Let K = 8, for no terribly good reason. - -Spec: - - [We can formalize this part once we think the design is a good one.] - -Acknowledgements: - - This design has been kicking around since Christian Grothoff and I came - up with it at PET 2004. (Nathan Evans, Christian Grothoff's student, - is working on implementing a fix based on this design in the summer - 2007 timeframe.) - diff --git a/doc/spec/proposals/111-local-traffic-priority.txt b/doc/spec/proposals/111-local-traffic-priority.txt deleted file mode 100644 index f8a37efc9..000000000 --- a/doc/spec/proposals/111-local-traffic-priority.txt +++ /dev/null @@ -1,153 +0,0 @@ -Filename: 111-local-traffic-priority.txt -Title: Prioritizing local traffic over relayed traffic -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 14-Mar-2007 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - We describe some ways to let Tor users operate as a relay and enforce - rate limiting for relayed traffic without impacting their locally - initiated traffic. - -Motivation: - - Right now we encourage people who use Tor as a client to configure it - as a relay too ("just click the button in Vidalia"). Most of these users - are on asymmetric links, meaning they have a lot more download capacity - than upload capacity. But if they enable rate limiting too, suddenly - they're limited to the same download capacity as upload capacity. And - they have to enable rate limiting, or their upstream pipe gets filled - up, starts dropping packets, and now their net connection doesn't work - even for non-Tor stuff. So they end up turning off the relaying part - so they can use Tor (and other applications) again. - - So far this hasn't mattered that much: most of our fast relays are - being operated only in relay mode, so the rate limiting makes sense - for them. But if we want to be able to attract many more relays in - the future, we need to let ordinary users act as relays too. - - Further, as we begin to deploy the blocking-resistance design and we - rely on ordinary users to click the "Tor for Freedom" button, this - limitation will become a serious stumbling block to getting volunteers - to act as bridges. - -The problem: - - Tor implements its rate limiting on the 'read' side by only reading - a certain number of bytes from the network in each second. If it has - emptied its token bucket, it doesn't read any more from the network; - eventually TCP notices and stalls until we resume reading. But if we - want to have two classes of service, we can't know what class a given - incoming cell will be until we look at it, at which point we've already - read it. - -Some options: - - Option 1: read when our token bucket is full enough, and if it turns - out that what we read was local traffic, then add the tokens back into - the token bucket. This will work when local traffic load alternates - with relayed traffic load; but it's a poor option in general, because - when we're receiving both local and relayed traffic, there are plenty - of cases where we'll end up with an empty token bucket, and then we're - back where we were before. - - More generally, notice that our problem is easy when a given TCP - connection either has entirely local circuits or entirely relayed - circuits. In fact, even if they are both present, if one class is - entirely idle (none of its circuits have sent or received in the past - N seconds), we can ignore that class until it wakes up again. So it - only gets complex when a single connection contains active circuits - of both classes. - - Next, notice that local traffic uses only the entry guards, whereas - relayed traffic likely doesn't. So if we're a bridge handling just - a few users, the expected number of overlapping connections would be - almost zero, and even if we're a full relay the number of overlapping - connections will be quite small. - - Option 2: build separate TCP connections for local traffic and for - relayed traffic. In practice this will actually only require a few - extra TCP connections: we would only need redundant TCP connections - to at most the number of entry guards in use. - - However, this approach has some drawbacks. First, if the remote side - wants to extend a circuit to you, how does it know which TCP connection - to send it on? We would need some extra scheme to label some connections - "client-only" during construction. Perhaps we could do this by seeing - whether any circuit was made via CREATE_FAST; but this still opens - up a race condition where the other side sends a create request - immediately. The only ways I can imagine to avoid the race entirely - are to specify our preference in the VERSIONS cell, or to add some - sort of "nope, not this connection, why don't you try another rather - than failing" response to create cells, or to forbid create cells on - connections that you didn't initiate and on which you haven't seen - any circuit creation requests yet -- this last one would lead to a bit - more connection bloat but doesn't seem so bad. And we already accept - this race for the case where directory authorities establish new TCP - connections periodically to check reachability, and then hope to hang - up on them soon after. (In any case this issue is moot for bridges, - since each destination will be one-way with respect to extend requests: - either receiving extend requests from bridge users or sending extend - requests to the Tor server, never both.) - - The second problem with option 2 is that using two TCP connections - reveals that there are two classes of traffic (and probably quickly - reveals which is which, based on throughput). Now, it's unclear whether - this information is already available to the other relay -- he would - easily be able to tell that some circuits are fast and some are rate - limited, after all -- but it would be nice to not add even more ways to - leak that information. Also, it's less clear that an external observer - already has this information if the circuits are all bundled together, - and for this case it's worth trying to protect it. - - Option 3: tell the other side about our rate limiting rules. When we - establish the TCP connection, specify the different policy classes we - have configured. Each time we extend a circuit, specify which policy - class that circuit should be part of. Then hope the other side obeys - our wishes. (If he doesn't, hang up on him.) Besides the design and - coordination hassles involved in this approach, there's a big problem: - our rate limiting classes apply to all our connections, not just - pairwise connections. How does one server we're connected to know how - much of our bucket has already been spent by another? I could imagine - a complex and inefficient "ok, now you can send me those two more cells - that you've got queued" protocol. I'm not sure how else we could do it. - - (Gosh. How could UDP designs possibly be compatible with rate limiting - with multiple bucket sizes?) - - Option 4: put both classes of circuits over a single connection, and - keep track of the last time we read or wrote a high-priority cell. If - it's been less than N seconds, give the whole connection high priority, - else give the whole connection low priority. - - Option 5: put both classes of circuits over a single connection, and - play a complex juggling game by periodically telling the remote side - what rate limits to set for that connection, so you end up giving - priority to the right connections but still stick to roughly your - intended bandwidthrate and relaybandwidthrate. - - Option 6: ? - -Prognosis: - - Nick really didn't like option 2 because of the partitioning questions. - - I've put option 4 into place as of Tor 0.2.0.3-alpha. - - In terms of implementation, it will be easy: just add a time_t to - or_connection_t that specifies client_used (used by the initiator - of the connection to rate limit it differently depending on how - recently the time_t was reset). We currently update client_used - in three places: - - command_process_relay_cell() when we receive a relay cell for - an origin circuit. - - relay_send_command_from_edge() when we send a relay cell for - an origin circuit. - - circuit_deliver_create_cell() when send a create cell. - We could probably remove the third case and it would still work, - but hey. - diff --git a/doc/spec/proposals/112-bring-back-pathlencoinweight.txt b/doc/spec/proposals/112-bring-back-pathlencoinweight.txt deleted file mode 100644 index e7cc6b4e3..000000000 --- a/doc/spec/proposals/112-bring-back-pathlencoinweight.txt +++ /dev/null @@ -1,165 +0,0 @@ -Filename: 112-bring-back-pathlencoinweight.txt -Title: Bring Back Pathlen Coin Weight -Version: $Revision$ -Last-Modified: $Date$ -Author: Mike Perry -Created: -Status: Superseded -Superseded-By: 115 - - -Overview: - - The idea is that users should be able to choose a weight which - probabilistically chooses their path lengths to be 2 or 3 hops. This - weight will essentially be a biased coin that indicates an - additional hop (beyond 2) with probability P. The user should be - allowed to choose 0 for this weight to always get 2 hops and 1 to - always get 3. - - This value should be modifiable from the controller, and should be - available from Vidalia. - - -Motivation: - - The Tor network is slow and overloaded. Increasingly often I hear - stories about friends and friends of friends who are behind firewalls, - annoying censorware, or under surveillance that interferes with their - productivity and Internet usage, or chills their speech. These people - know about Tor, but they choose to put up with the censorship because - Tor is too slow to be usable for them. In fact, to download a fresh, - complete copy of levine-timing.pdf for the Anonymity Implications - section of this proposal over Tor took me 3 tries. - - There are many ways to improve the speed problem, and of course we - should and will implement as many as we can. Johannes's GSoC project - and my reputation system are longer term, higher-effort things that - will still provide benefit independent of this proposal. - - However, reducing the path length to 2 for those who do not need the - (questionable) extra anonymity 3 hops provide not only improves - their Tor experience but also reduces their load on the Tor network by - 33%, and can be done in less than 10 lines of code. That's not just - Win-Win, it's Win-Win-Win. - - Furthermore, when blocking resistance measures insert an extra relay - hop into the equation, 4 hops will certainly be completely unusable - for these users, especially since it will be considerably more - difficult to balance the load across a dark relay net than balancing - the load on Tor itself (which today is still not without its flaws). - - -Anonymity Implications: - - It has long been established that timing attacks against mixed - networks are extremely effective, and that regardless of path - length, if the adversary has compromised your first and last - hop of your path, you can assume they have compromised your - identity for that connection. - - In [1], it is demonstrated that for all but the slowest, lossiest - networks, error rates for false positives and false negatives were - very near zero. Only for constant streams of traffic over slow and - (more importantly) extremely lossy network links did the error rate - hit 20%. For loss rates typical to the Internet, even the error rate - for slow nodes with constant traffic streams was 13%. - - When you take into account that most Tor streams are not constant, - but probably much more like their "HomeIP" dataset, which consists - mostly of web traffic that exists over finite intervals at specific - times, error rates drop to fractions of 1%, even for the "worst" - network nodes. - - Therefore, the user has little benefit from the extra hop, assuming - the adversary does timing correlation on their nodes. The real - protection is the probability of getting both the first and last hop, - and this is constant whether the client chooses 2 hops, 3 hops, or 42. - - Partitioning attacks form another concern. Since Tor uses telescoping - to build circuits, it is possible to tell a user is constructing only - two hop paths at the entry node. It is questionable if this data is - actually worth anything though, especially if the majority of users - have easy access to this option, and do actually choose their path - lengths semi-randomly. - - Nick has postulated that exits may also be able to tell that you are - using only 2 hops by the amount of time between sending their - RELAY_CONNECTED cell and the first bit of RELAY_DATA traffic they - see from the OP. I doubt that they will be able to make much use - of this timing pattern, since it will likely vary widely depending - upon the type of node selected for that first hop, and the user's - connection rate to that first hop. It is also questionable if this - data is worth anything, especially if many users are using this - option (and I imagine many will). - - Perhaps most seriously, two hop paths do allow malicious guards - to easily fail circuits if they do not extend to their colluding peers - for the exit hop. Since guards can detect the number of hops in a - path, they could always fail the 3 hop circuits and focus on - selectively failing the two hop ones until a peer was chosen. - - I believe currently guards are rotated if circuits fail, which does - provide some protection, but this could be changed so that an entry - guard is completely abandoned after a certain ratio of extend or - general circuit failures with respect to non-failed circuits. This - could possibly be gamed to increase guard turnover, but such a game - would be much more noticeable than an individual guard failing circuits, - though, since it would affect all clients, not just those who chose - a particular guard. - - -Why not fix Pathlen=2?: - - The main reason I am not advocating that we always use 2 hops is that - in some situations, timing correlation evidence by itself may not be - considered as solid and convincing as an actual, uninterrupted, fully - traced path. Are these timing attacks as effective on a real network - as they are in simulation? Would an extralegal adversary or authoritarian - government even care? In the face of these situation-dependent unknowns, - it should be up to the user to decide if this is a concern for them or not. - - It should probably also be noted that even a false positive - rate of 1% for a 200k concurrent-user network could mean that for a - given node, a given stream could be confused with something like 10 - users, assuming ~200 nodes carry most of the traffic (ie 1000 users - each). Though of course to really know for sure, someone needs to do - an attack on a real network, unfortunately. - - -Implementation: - - new_route_len() can be modified directly with a check of the - PathlenCoinWeight option (converted to percent) and a call to - crypto_rand_int(0,100) for the weighted coin. - - The entry_guard_t structure could have num_circ_failed and - num_circ_succeeded members such that if it exceeds N% circuit - extend failure rate to a second hop, it is removed from the entry list. - N should be sufficiently high to avoid churn from normal Tor circuit - failure as determined by TorFlow scans. - - The Vidalia option should be presented as a boolean, to minimize confusion - for the user. Something like a radiobutton with: - - * "I use Tor for Censorship Resistance, not Anonymity. Speed is more - important to me than Anonymity." - * "I use Tor for Anonymity. I need extra protection at the cost of speed." - - and then some explanation in the help for exactly what this means, and - the risks involved with eliminating the adversary's need for timing attacks - wrt to false positives, etc. - -Migration: - - Phase one: Experiment with the proper ratio of circuit failures - used to expire garbage or malicious guards via TorFlow. - - Phase two: Re-enable config and modify new_route_len() to add an - extra hop if coin comes up "heads". - - Phase three: Make radiobutton in Vidalia, along with help entry - that explains in layman's terms the risks involved. - - -[1] http://www.cs.umass.edu/~mwright/papers/levine-timing.pdf diff --git a/doc/spec/proposals/113-fast-authority-interface.txt b/doc/spec/proposals/113-fast-authority-interface.txt deleted file mode 100644 index 20cf33e42..000000000 --- a/doc/spec/proposals/113-fast-authority-interface.txt +++ /dev/null @@ -1,87 +0,0 @@ -Filename: 113-fast-authority-interface.txt -Title: Simplifying directory authority administration -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: -Status: Superseded - -Overview - -The problem: - - Administering a directory authority is a pain: you need to go through - emails and manually add new nodes as "named". When bad things come up, - you need to mark nodes (or whole regions) as invalid, badexit, etc. - - This means that mostly, authority admins don't: only 2/4 current authority - admins actually bind names or list bad exits, and those two have often - complained about how annoying it is to do so. - - Worse, name binding is a common path, but it's a pain in the neck: nobody - has done it for a couple of months. - -Digression: who knows what? - - It's trivial for Tor to automatically keep track of all of the - following information about a server: - name, fingerprint, IP, last-seen time, first-seen time, declared - contact. - - All we need to have the administrator set is: - - Is this name/fingerprint pair bound? - - Is this fingerprint/IP a bad exit? - - Is this fingerprint/IP an invalid node? - - Is this fingerprint/IP to be rejected? - - The workflow for authority admins has two parts: - - Periodically, go through tor-ops and add new names. This doesn't - need to be done urgently. - - Less often, mark badly behaved serves as badly behaved. This is more - urgent. - -Possible solution #1: Web-interface for name binding. - - Deprecate use of the tor-ops mailing list; instead, have operators go to a - webform and enter their server info. This would put the information in a - standardized format, thus allowing quick, nearly-automated approval and - reply. - -Possible solution #2: Self-binding names. - - Peter Palfrader has proposed that names be assigned automatically to nodes - that have been up and running and valid for a while. - -Possible solution #3: Self-maintaining approved-routers file - - Mixminion alpha has a neat feature where whenever a new server is seen, - a stub line gets added to a configuration file. For Tor, it could look - something like this: - - ## First seen with this key on 2007-04-21 13:13:14 - ## Stayed up for at least 12 hours on IP 192.168.10.10 - #RouterName AAAABBBBCCCCDDDDEFEF - - (Note that the implementation needs to parse commented lines to make sure - that it doesn't add duplicates, but that's not so hard.) - - To add a router as named, administrators would only need to uncomment the - entry. This automatically maintained file could be kept separately from a - manually maintained one. - - This could be combined with solution #2, such that Tor would do the hard - work of uncommenting entries for routers that should get Named, but - operators could override its decisions. - -Possible solution #4: A separate mailing list for authority operators. - - Right now, the tor-ops list is very high volume. There should be another - list that's only for dealing with problems that need prompt action, like - marking a router as !badexit. - -Resolution: - - Solution #2 is described in "Proposal 123: Naming authorities - automatically create bindings", and that approach is implemented. - There are remaining issues in the problem statement above that need - their own solutions. diff --git a/doc/spec/proposals/114-distributed-storage.txt b/doc/spec/proposals/114-distributed-storage.txt deleted file mode 100644 index e9271fb82..000000000 --- a/doc/spec/proposals/114-distributed-storage.txt +++ /dev/null @@ -1,441 +0,0 @@ -Filename: 114-distributed-storage.txt -Title: Distributed Storage for Tor Hidden Service Descriptors -Version: $Revision$ -Last-Modified: $Date$ -Author: Karsten Loesing -Created: 13-May-2007 -Status: Closed -Implemented-In: 0.2.0.x - -Change history: - - 13-May-2007 Initial proposal - 14-May-2007 Added changes suggested by Lasse Øverlier - 30-May-2007 Changed descriptor format, key length discussion, typos - 09-Jul-2007 Incorporated suggestions by Roger, added status of specification - and implementation for upcoming GSoC mid-term evaluation - 11-Aug-2007 Updated implementation statuses, included non-consecutive - replication to descriptor format - 20-Aug-2007 Renamed config option HSDir as HidServDirectoryV2 - 02-Dec-2007 Closed proposal - -Overview: - - The basic idea of this proposal is to distribute the tasks of storing and - serving hidden service descriptors from currently three authoritative - directory nodes among a large subset of all onion routers. The three - reasons to do this are better robustness (availability), better - scalability, and improved security properties. Further, - this proposal suggests changes to the hidden service descriptor format to - prevent new security threats coming from decentralization and to gain even - better security properties. - -Status: - - As of December 2007, the new hidden service descriptor format is implemented - and usable. However, servers and clients do not yet make use of descriptor - cookies, because there are open usability issues of this feature that might - be resolved in proposal 121. Further, hidden service directories do not - perform replication by themselves, because (unauthorized) replica fetch - requests would allow any attacker to fetch all hidden service descriptors in - the system. As neither issue is critical to the functioning of v2 - descriptors and their distribution, this proposal is considered as Closed. - -Motivation: - - The current design of hidden services exhibits the following performance and - security problems: - - First, the three hidden service authoritative directories constitute a - performance bottleneck in the system. The directory nodes are responsible for - storing and serving all hidden service descriptors. As of May 2007 there are - about 1000 descriptors at a time, but this number is assumed to increase in - the future. Further, there is no replication protocol for descriptors between - the three directory nodes, so that hidden services must ensure the - availability of their descriptors by manually publishing them on all - directory nodes. Whenever a fourth or fifth hidden service authoritative - directory is added, hidden services will need to maintain an equally - increasing number of replicas. These scalability issues have an impact on the - current usage of hidden services and put an even higher burden on the - development of new kinds of applications for hidden services that might - require storing even more descriptors. - - Second, besides posing a limitation to scalability, storing all hidden - service descriptors on three directory nodes also constitutes a security - risk. The directory node operators could easily analyze the publish and fetch - requests to derive information on service activity and usage and read the - descriptor contents to determine which onion routers work as introduction - points for a given hidden service and need to be attacked or threatened to - shut it down. Furthermore, the contents of a hidden service descriptor offer - only minimal security properties to the hidden service. Whoever gets aware of - the service ID can easily find out whether the service is active at the - moment and which introduction points it has. This applies to (former) - clients, (former) introduction points, and of course to the directory nodes. - It requires only to request the descriptor for the given service ID, which - can be performed by anyone anonymously. - - This proposal suggests two major changes to approach the described - performance and security problems: - - The first change affects the storage location for hidden service descriptors. - Descriptors are distributed among a large subset of all onion routers instead - of three fixed directory nodes. Each storing node is responsible for a subset - of descriptors for a limited time only. It is not able to choose which - descriptors it stores at a certain time, because this is determined by its - onion ID which is hard to change frequently and in time (only routers which - are stable for a given time are accepted as storing nodes). In order to - resist single node failures and untrustworthy nodes, descriptors are - replicated among a certain number of storing nodes. A first replication - protocol makes sure that descriptors don't get lost when the node population - changes; therefore, a storing node periodically requests the descriptors from - its siblings. A second replication protocol distributes descriptors among - non-consecutive nodes of the ID ring to prevent a group of adversaries from - generating new onion keys until they have consecutive IDs to create a 'black - hole' in the ring and make random services unavailable. Connections to - storing nodes are established by extending existing circuits by one hop to - the storing node. This also ensures that contents are encrypted. The effect - of this first change is that the probability that a single node operator - learns about a certain hidden service is very small and that it is very hard - to track a service over time, even when it collaborates with other node - operators. - - The second change concerns the content of hidden service descriptors. - Obviously, security problems cannot be solved only by decentralizing storage; - in fact, they could also get worse if done without caution. At first, a - descriptor ID needs to change periodically in order to be stored on changing - nodes over time. Next, the descriptor ID needs to be computable only for the - service's clients, but should be unpredictable for all other nodes. Further, - the storing node needs to be able to verify that the hidden service is the - true originator of the descriptor with the given ID even though it is not a - client. Finally, a storing node should learn as little information as - necessary by storing a descriptor, because it might not be as trustworthy as - a directory node; for example it does not need to know the list of - introduction points. Therefore, a second key is applied that is only known to - the hidden service provider and its clients and that is not included in the - descriptor. It is used to calculate descriptor IDs and to encrypt the - introduction points. This second key can either be given to all clients - together with the hidden service ID, or to a group or a single client as - an authentication token. In the future this second key could be the result of - some key agreement protocol between the hidden service and one or more - clients. A new text-based format is proposed for descriptors instead of an - extension of the existing binary format for reasons of future extensibility. - -Design: - - The proposed design is described by the required changes to the current - design. These requirements are grouped by content, rather than by affected - specification documents or code files, and numbered for reference below. - - Hidden service clients, servers, and directories: - - /1/ Create routing list - - All participants can filter the consensus status document received from the - directory authorities to one routing list containing only those servers - that store and serve hidden service descriptors and which are running for - at least 24 hours. A participant only trusts its own routing list and never - learns about routing information from other parties. - - /2/ Determine responsible hidden service directory - - All participants can determine the hidden service directory that is - responsible for storing and serving a given ID, as well as the hidden - service directories that replicate its content. Every hidden service - directory is responsible for the descriptor IDs in the interval from - its predecessor, exclusive, to its own ID, inclusive. Further, a hidden - service directory holds replicas for its n predecessors, where n denotes - the number of consecutive replicas. (requires /1/) - - [/3/ and /4/ were requirements to use BEGIN_DIR cells for directory - requests which have not been fulfilled in the course of the implementation - of this proposal, but elsewhere.] - - Hidden service directory nodes: - - /5/ Advertise hidden service directory functionality - - Every onion router that has its directory port open can decide whether it - wants to store and serve hidden service descriptors by setting a new config - option "HidServDirectoryV2" 0|1 to 1. An onion router with this config - option being set includes the flag "hidden-service-dir" in its router - descriptors that it sends to directory authorities. - - /6/ Accept v2 publish requests, parse and store v2 descriptors - - Hidden service directory nodes accept publish requests for hidden service - descriptors and store them to their local memory. (It is not necessary to - make descriptors persistent, because after disconnecting, the onion router - would not be accepted as storing node anyway, because it has not been - running for at least 24 hours.) All requests and replies are formatted as - HTTP messages. Requests are directed to the router's directory port and are - contained within BEGIN_DIR cells. A hidden service directory node stores a - descriptor only when it thinks that it is responsible for storing that - descriptor based on its own routing table. Every hidden service directory - node is responsible for the descriptor IDs in the interval of its n-th - predecessor in the ID circle up to its own ID (n denotes the number of - consecutive replicas). (requires /1/) - - /7/ Accept v2 fetch requests - - Same as /6/, but with fetch requests for hidden service descriptors. - (requires /2/) - - /8/ Replicate descriptors with neighbors - - A hidden service directory node replicates descriptors from its two - predecessors by downloading them once an hour. Further, it checks its - routing table periodically for changes. Whenever it realizes that a - predecessor has left the network, it establishes a connection to the new - n-th predecessor and requests its stored descriptors in the interval of its - (n+1)-th predecessor and the requested n-th predecessor. Whenever it - realizes that a new onion router has joined with an ID higher than its - former n-th predecessor, it adds it to its predecessors and discards all - descriptors in the interval of its (n+1)-th and its n-th predecessor. - (requires /1/) - - [Dec 02: This function has not been implemented, because arbitrary nodes - what have been able to download the entire set of v2 descriptors. An - authorized replication request would be necessary. For the moment, the - system runs without any directory-side replication. -KL] - - Authoritative directory nodes: - - /9/ Confirm a router's hidden service directory functionality - - Directory nodes include a new flag "HSDir" for routers that decided to - provide storage for hidden service descriptors and that are running for at - least 24 hours. The last requirement prevents a node from frequently - changing its onion key to become responsible for an identifier it wants to - target. - - Hidden service provider: - - /10/ Configure v2 hidden service - - Each hidden service provider that has set the config option - "PublishV2HidServDescriptors" 0|1 to 1 is configured to publish v2 - descriptors and conform to the v2 connection establishment protocol. When - configuring a hidden service, a hidden service provider checks if it has - already created a random secret_cookie and a hostname2 file; if not, it - creates both of them. (requires /2/) - - /11/ Establish introduction points with fresh key - - If configured to publish only v2 descriptors and no v0/v1 descriptors any - more, a hidden service provider that is setting up the hidden service at - introduction points does not pass its own public key, but the public key - of a freshly generated key pair. It also includes these fresh public keys - in the hidden service descriptor together with the other introduction point - information. The reason is that the introduction point does not need to and - therefore should not know for which hidden service it works, so as to - prevent it from tracking the hidden service's activity. (If a hidden - service provider supports both, v0/v1 and v2 descriptors, v0/v1 clients - rely on the fact that all introduction points accept the same public key, - so that this new feature cannot be used.) - - /12/ Encode v2 descriptors and send v2 publish requests - - If configured to publish v2 descriptors, a hidden service provider - publishes a new descriptor whenever its content changes or a new - publication period starts for this descriptor. If the current publication - period would only last for less than 60 minutes (= 2 x 30 minutes to allow - the server to be 30 minutes behind and the client 30 minutes ahead), the - hidden service provider publishes both a current descriptor and one for - the next period. Publication is performed by sending the descriptor to all - hidden service directories that are responsible for keeping replicas for - the descriptor ID. This includes two non-consecutive replicas that are - stored at 3 consecutive nodes each. (requires /1/ and /2/) - - Hidden service client: - - /13/ Send v2 fetch requests - - A hidden service client that has set the config option - "FetchV2HidServDescriptors" 0|1 to 1 handles SOCKS requests for v2 onion - addresses by requesting a v2 descriptor from a randomly chosen hidden - service directory that is responsible for keeping replica for the - descriptor ID. In total there are six replicas of which the first and the - last three are stored on consecutive nodes. The probability of picking one - of the three consecutive replicas is 1/6, 2/6, and 3/6 to incorporate the - fact that the availability will be the highest on the node with next higher - ID. A hidden service client relies on the hidden service provider to store - two sets of descriptors to compensate clock skew between service and - client. (requires /1/ and /2/) - - /14/ Process v2 fetch reply and parse v2 descriptors - - A hidden service client that has sent a request for a v2 descriptor can - parse it and store it to the local cache of rendezvous service descriptors. - - /15/ Establish connection to v2 hidden service - - A hidden service client can establish a connection to a hidden service - using a v2 descriptor. This includes using the secret cookie for decrypting - the introduction points contained in the descriptor. When contacting an - introduction point, the client does not use the public key of the hidden - service provider, but the freshly-generated public key that is included in - the hidden service descriptor. Whether or not a fresh key is used instead - of the key of the hidden service depends on the available protocol versions - that are included in the descriptor; by this, connection establishment is - to a certain extend decoupled from fetching the descriptor. - - Hidden service descriptor: - - (Requirements concerning the descriptor format are contained in /6/ and /7/.) - - The new v2 hidden service descriptor format looks like this: - - onion-address = h(public-key) + cookie - descriptor-id = h(h(public-key) + h(time-period + cookie + relica)) - descriptor-content = { - descriptor-id, - version, - public-key, - h(time-period + cookie + replica), - timestamp, - protocol-versions, - { introduction-points } encrypted with cookie - } signed with private-key - - The "descriptor-id" needs to change periodically in order for the - descriptor to be stored on changing nodes over time. It may only be - computable by a hidden service provider and all of his clients to prevent - unauthorized nodes from tracking the service activity by periodically - checking whether there is a descriptor for this service. Finally, the - hidden service directory needs to be able to verify that the hidden service - provider is the true originator of the descriptor with the given ID. - - Therefore, "descriptor-id" is derived from the "public-key" of the hidden - service provider, the current "time-period" which changes every 24 hours, - a secret "cookie" shared between hidden service provider and clients, and - a "replica" denoting the number of this non-consecutive replica. (The - "time-period" is constructed in a way that time periods do not change at - the same moment for all descriptors by deriving a value between 0:00 and - 23:59 hours from h(public-key) and making the descriptors of this hidden - service provider expire at that time of the day.) The "descriptor-id" is - defined to be 160 bits long. [extending the "descriptor-id" length - suggested by LØ] - - Only the hidden service provider and the clients are able to generate - future "descriptor-ID"s. Hence, the "onion-address" is extended from now - the hash value of "public-key" by the secret "cookie". The "public-key" is - determined to be 80 bits long, whereas the "cookie" is dimensioned to be - 120 bits long. This makes a total of 200 bits or 40 base32 chars, which is - quite a lot to handle for a human, but necessary to provide sufficient - protection against an adversary from generating a key pair with same - "public-key" hash or guessing the "cookie". - - A hidden service directory can verify that a descriptor was created by the - hidden service provider by checking if the "descriptor-id" corresponds to - the "public-key" and if the signature can be verified with the - "public-key". - - The "introduction-points" that are included in the descriptor are encrypted - using the same "cookie" that is shared between hidden service provider and - clients. [correction to use another key than h(time-period + cookie) as - encryption key for introduction points made by LØ] - - A new text-based format is proposed for descriptors instead of an extension - of the existing binary format for reasons of future extensibility. - -Security implications: - - The security implications of the proposed changes are grouped by the roles of - nodes that could perform attacks or on which attacks could be performed. - - Attacks by authoritative directory nodes - - Authoritative directory nodes are no longer the single places in the - network that know about a hidden service's activity and introduction - points. Thus, they cannot perform attacks using this information, e.g. - track a hidden service's activity or usage pattern or attack its - introduction points. Formerly, it would only require a single corrupted - authoritative directory operator to perform such an attack. - - Attacks by hidden service directory nodes - - A hidden service directory node could misuse a stored descriptor to track a - hidden service's activity and usage pattern by clients. Though there is no - countermeasure against this kind of attack, it is very expensive to track a - certain hidden service over time. An attacker would need to run a large - number of stable onion routers that work as hidden service directory nodes - to have a good probability to become responsible for its changing - descriptor IDs. For each period, the probability is: - - 1-(N-c choose r)/(N choose r) for N-c>=r and 1 otherwise, with N - as total - number of hidden service directories, c as compromised nodes, and r as - number of replicas - - The hidden service directory nodes could try to make a certain hidden - service unavailable to its clients. Therefore, they could discard all - stored descriptors for that hidden service and reply to clients that there - is no descriptor for the given ID or return an old or false descriptor - content. The client would detect a false descriptor, because it could not - contain a correct signature. But an old content or an empty reply could - confuse the client. Therefore, the countermeasure is to replicate - descriptors among a small number of hidden service directories, e.g. 5. - The probability of a group of collaborating nodes to make a hidden service - completely unavailable is in each period: - - (c choose r)/(N choose r) for c>=r and N>=r, and 0 otherwise, - with N as total - number of hidden service directories, c as compromised nodes, and r as - number of replicas - - A hidden service directory could try to find out which introduction points - are working on behalf of a hidden service. In contrast to the previous - design, this is not possible anymore, because this information is encrypted - to the clients of a hidden service. - - Attacks on hidden service directory nodes - - An anonymous attacker could try to swamp a hidden service directory with - false descriptors for a given descriptor ID. This is prevented by requiring - that descriptors are signed. - - Anonymous attackers could swamp a hidden service directory with correct - descriptors for non-existing hidden services. There is no countermeasure - against this attack. However, the creation of valid descriptors is more - expensive than verification and storage in local memory. This should make - this kind of attack unattractive. - - Attacks by introduction points - - Current or former introduction points could try to gain information on the - hidden service they serve. But due to the fresh key pair that is used by - the hidden service, this attack is not possible anymore. - - Attacks by clients - - Current or former clients could track a hidden service's activity, attack - its introduction points, or determine the responsible hidden service - directory nodes and attack them. There is nothing that could prevent them - from doing so, because honest clients need the full descriptor content to - establish a connection to the hidden service. At the moment, the only - countermeasure against dishonest clients is to change the secret cookie and - pass it only to the honest clients. - -Compatibility: - - The proposed design is meant to replace the current design for hidden service - descriptors and their storage in the long run. - - There should be a first transition phase in which both, the current design - and the proposed design are served in parallel. Onion routers should start - serving as hidden service directories, and hidden service providers and - clients should make use of the new design if both sides support it. Hidden - service providers should be allowed to publish descriptors of the current - format in parallel, and authoritative directories should continue storing and - serving these descriptors. - - After the first transition phase, hidden service providers should stop - publishing descriptors on authoritative directories, and hidden service - clients should not try to fetch descriptors from the authoritative - directories. However, the authoritative directories should continue serving - hidden service descriptors for a second transition phase. As of this point, - all v2 config options should be set to a default value of 1. - - After the second transition phase, the authoritative directories should stop - serving hidden service descriptors. - diff --git a/doc/spec/proposals/115-two-hop-paths.txt b/doc/spec/proposals/115-two-hop-paths.txt deleted file mode 100644 index ee10d949c..000000000 --- a/doc/spec/proposals/115-two-hop-paths.txt +++ /dev/null @@ -1,387 +0,0 @@ -Filename: 115-two-hop-paths.txt -Title: Two Hop Paths -Version: $Revision$ -Last-Modified: $Date$ -Author: Mike Perry -Created: -Status: Dead -Supersedes: 112 - - -Overview: - - The idea is that users should be able to choose if they would like - to have either two or three hop paths through the tor network. - - Let us be clear: the users who would choose this option should be - those that are concerned with IP obfuscation only: ie they would not be - targets of a resource-intensive multi-node attack. It is sometimes said - that these users should find some other network to use other than Tor. - This is a foolish suggestion: more users improves security of everyone, - and the current small userbase size is a critical hindrance to - anonymity, as is discussed below and in [1]. - - This value should be modifiable from the controller, and should be - available from Vidalia. - - -Motivation: - - The Tor network is slow and overloaded. Increasingly often I hear - stories about friends and friends of friends who are behind firewalls, - annoying censorware, or under surveillance that interferes with their - productivity and Internet usage, or chills their speech. These people - know about Tor, but they choose to put up with the censorship because - Tor is too slow to be usable for them. In fact, to download a fresh, - complete copy of levine-timing.pdf for the Theoretical Argument - section of this proposal over Tor took me 3 tries. - - Furthermore, the biggest current problem with Tor's anonymity for - those who really need it is not someone attacking the network to - discover who they are. It's instead the extreme danger that so few - people use Tor because it's so slow, that those who do use it have - essentially no confusion set. - - The recent case where the professor and the rogue Tor user were the - only Tor users on campus, and thus suspected in an incident involving - Tor and that University underscores this point: "That was why the police - had come to see me. They told me that only two people on our campus were - using Tor: me and someone they suspected of engaging in an online scam. - The detectives wanted to know whether the other user was a former - student of mine, and why I was using Tor"[1]. - - Not only does Tor provide no anonymity if you use it to be anonymous - but are obviously from a certain institution, location or circumstance, - it is also dangerous to use Tor for risk of being accused of having - something significant enough to hide to be willing to put up with - the horrible performance as opposed to using some weaker alternative. - - There are many ways to improve the speed problem, and of course we - should and will implement as many as we can. Johannes's GSoC project - and my reputation system are longer term, higher-effort things that - will still provide benefit independent of this proposal. - - However, reducing the path length to 2 for those who do not need the - extra anonymity 3 hops provide not only improves their Tor experience - but also reduces their load on the Tor network by 33%, and should - increase adoption of Tor by a good deal. That's not just Win-Win, it's - Win-Win-Win. - - -Who will enable this option? - - This is the crux of the proposal. Admittedly, there is some anonymity - loss and some degree of decreased investment required on the part of - the adversary to attack 2 hop users versus 3 hop users, even if it is - minimal and limited mostly to up-front costs and false positives. - - The key questions are: - - 1. Are these users in a class such that their risk is significantly - less than the amount of this anonymity loss? - - 2. Are these users able to identify themselves? - - Many many users of Tor are not at risk for an adversary capturing c/n - nodes of the network just to see what they do. These users use Tor to - circumvent aggressive content filters, or simply to keep their IP out of - marketing and search engine databases. Most content filters have no - interest in running Tor nodes to catch violators, and marketers - certainly would never consider such a thing, both on a cost basis and a - legal one. - - In a sense, this represents an alternate threat model against these - users who are not at risk for Tor's normal threat model. - - It should be evident to these users that they fall into this class. All - that should be needed is a radio button - - * "I use Tor for local content filter circumvention and/or IP obfuscation, - not anonymity. Speed is more important to me than high anonymity. - No one will make considerable efforts to determine my real IP." - * "I use Tor for anonymity and/or national-level, legally enforced - censorship. It is possible effort will be taken to identify - me, including but not limited to network surveillance. I need more - protection." - - and then some explanation in the help for exactly what this means, and - the risks involved with eliminating the adversary's need for timing - attacks with respect to false positives. Ultimately, the decision is a - simple one that can be made without this information, however. The user - does not need Paul Syverson to instruct them on the deep magic of Onion - Routing to make this decision. They just need to know why they use Tor. - If they use it just to stay out of marketing databases and/or bypass a - local content filter, two hops is plenty. This is likely the vast - majority of Tor users, and many non-users we would like to bring on - board. - - So, having established this class of users, let us now go on to - examine theoretical and practical risks we place them at, and determine - if these risks violate the users needs, or introduce additional risk - to node operators who may be subject to requests from law enforcement - to track users who need 3 hops, but use 2 because they enjoy the - thrill of russian roulette. - - -Theoretical Argument: - - It has long been established that timing attacks against mixed - and onion networks are extremely effective, and that regardless - of path length, if the adversary has compromised your first and - last hop of your path, you can assume they have compromised your - identity for that connection. - - In fact, it was demonstrated that for all but the slowest, lossiest - networks, error rates for false positives and false negatives were - very near zero[2]. Only for constant streams of traffic over slow and - (more importantly) extremely lossy network links did the error rate - hit 20%. For loss rates typical to the Internet, even the error rate - for slow nodes with constant traffic streams was 13%. - - When you take into account that most Tor streams are not constant, - but probably much more like their "HomeIP" dataset, which consists - mostly of web traffic that exists over finite intervals at specific - times, error rates drop to fractions of 1%, even for the "worst" - network nodes. - - Therefore, the user has little benefit from the extra hop, assuming - the adversary does timing correlation on their nodes. Since timing - correlation is simply an implementation issue and is most likely - a single up-front cost (and one that is like quite a bit cheaper - than the cost of the machines purchased to host the nodes to mount - an attack), the real protection is the low probability of getting - both the first and last hop of a client's stream. - - -Practical Issues: - - Theoretical issues aside, there are several practical issues with the - implementation of Tor that need to be addressed to ensure that - identity information is not leaked by the implementation. - - Exit policy issues: - - If a client chooses an exit with a very restrictive exit policy - (such as an IP or IP range), the first hop then knows a good deal - about the destination. For this reason, clients should not select - exits that match their destination IP with anything other than "*". - - Partitioning: - - Partitioning attacks form another concern. Since Tor uses telescoping - to build circuits, it is possible to tell a user is constructing only - two hop paths at the entry node and on the local network. An external - adversary can potentially differentiate 2 and 3 hop users, and decide - that all IP addresses connecting to Tor and using 3 hops have something - to hide, and should be scrutinized more closely or outright apprehended. - - One solution to this is to use the "leaky-circuit" method of attaching - streams: The user always creates 3-hop circuits, but if the option - is enabled, they always exit from their 2nd hop. The ideal solution - would be to create a RELAY_SHISHKABOB cell which contains onion - skins for every host along the path, but this requires protocol - changes at the nodes to support. - - Guard nodes: - - Since guard nodes can rotate due to client relocation, network - failure, node upgrades and other issues, if you amortize the risk a - mobile, dialup, or otherwise intermittently connected user is exposed to - over any reasonable duration of Tor usage (on the order of a year), it - is the same with or without guard nodes. Assuming an adversary has - c%/n% of network bandwidth, and guards rotate on average with period R, - statistically speaking, it's merely a question of if the user wishes - their risk to be concentrated with probability c/n over an expected - period of R*c, and probability 0 over an expected period of R*(n-c), - versus a continuous risk of (c/n)^2. So statistically speaking, guards - only create a time-tradeoff of risk over the long run for normal Tor - usage. Rotating guards do not reduce risk for normal client usage long - term.[3] - - On other other hand, assuming a more stable method of guard selection - and preservation is devised, or a more stable client side network than - my own is typical (which rotates guards frequently due to network issues - and moving about), guard nodes provide a tradeoff in the form of c/n% of - the users being "sacrificial users" who are exposed to high risk O(c/n) - of identification, while the rest of the network is exposed to zero - risk. - - The nature of Tor makes it likely an adversary will take a "shock and - awe" approach to suppressing Tor by rounding up a few users whose - browsing activity has been observed to be made into examples, in an - attempt to prove that Tor is not perfect. - - Since this "shock and awe" attack can be applied with or without guard - nodes, stable guard nodes do offer a measure of accountability of sorts. - If a user was using a small set of guard nodes and knows them well, and - then is suddenly apprehended as a result of Tor usage, having a fixed - set of entry points to suspect is a lot better than suspecting the whole - network. Conversely, it can also give non-apprehended users comfort - that they are likely to remain safe indefinitely with their set of (now - presumably trusted) guards. This is probably the most beneficial - property of reliable guards: they deter the adversary from mounting - "shock and awe" attacks because the surviving users will not - intimidated, but instead made more confident. Of course, guards need to - be made much more stable and users need to be encouraged to know their - guards for this property to really take effect. - - This beneficial property of client vigilance also carries over to an - active adversary, except in this case instead of relying on the user - to remember their guard nodes and somehow communicate them after - apprehension, the code can alert them to the presence of an active - adversary before they are apprehended. But only if they use guard nodes. - - So lets consider the active adversary: Two hop paths allow malicious - guards to get considerably more benefit from failing circuits if they do - not extend to their colluding peers for the exit hop. Since guards can - detect the number of hops in a path via either timing or by statistical - analysis of the exit policy of the 2nd hop, they can perform this attack - predominantly against 2 hop users. - - This can be addressed by completely abandoning an entry guard after a - certain ratio of extend or general circuit failures with respect to - non-failed circuits. The proper value for this ratio can be determined - experimentally with TorFlow. There is the possibility that the local - network can abuse this feature to cause certain guards to be dropped, - but they can do that anyways with the current Tor by just making guards - they don't like unreachable. With this mechanism, Tor will complain - loudly if any guard failure rate exceeds the expected in any failure - case, local or remote. - - Eliminating guards entirely would actually not address this issue due - to the time-tradeoff nature of risk. In fact, it would just make it - worse. Without guard nodes, it becomes much more difficult for clients - to become alerted to Tor entry points that are failing circuits to make - sure that they only devote bandwidth to carry traffic for streams which - they observe both ends. Yet the rogue entry points are still able to - significantly increase their success rates by failing circuits. - - For this reason, guard nodes should remain enabled for 2 hop users, - at least until an IP-independent, undetectable guard scanner can - be created. TorFlow can scan for failing guards, but after a while, - its unique behavior gives away the fact that its IP is a scanner and - it can be given selective service. - - Consideration of risks for node operators: - - There is a serious risk for two hop users in the form of guard - profiling. If an adversary running an exit node notices that a - particular site is always visited from a fixed previous hop, it is - likely that this is a two hop user using a certain guard, which could be - monitored to determine their identity. Thus, for the protection of both - 2 hop users and node operators, 2 hop users should limit their guard - duration to a sufficient number of days to verify reliability of a node, - but not much more. This duration can be determined experimentally by - TorFlow. - - Considering a Tor client builds on average 144 circuits/day (10 - minutes per circuit), if the adversary owns c/n% of exits on the - network, they can expect to see 144*c/n circuits from this user, or - about 14 minutes of usage per day per percentage of network penetration. - Since it will take several occurrences of user-linkable exit content - from the same predecessor hop for the adversary to have any confidence - this is a 2 hop user, it is very unlikely that any sort of demands made - upon the predecessor node would guaranteed to be effective (ie it - actually was a guard), let alone be executed in time to apprehend the - user before they rotated guards. - - The reverse risk also warrants consideration. If a malicious guard has - orders to surveil Mike Perry, it can determine Mike Perry is using two - hops by observing his tendency to choose a 2nd hop with a viable exit - policy. This can be done relatively quickly, unfortunately, and - indicates Mike Perry should spend some of his time building real 3 hop - circuits through the same guards, to require them to at least wait for - him to actually use Tor to determine his style of operation, rather than - collect this information from his passive building patterns. - - However, to actively determine where Mike Perry is going, the guard - will need to require logging ahead of time at multiple exit nodes that - he may use over the course of the few days while he is at that guard, - and correlate the usage times of the exit node with Mike Perry's - activity at that guard for the few days he uses it. At this point, the - adversary is mounting a scale and method of attack (widespread logging, - timing attacks) that works pretty much just as effectively against 3 - hops, so exit node operators are exposed to no additional danger than - they otherwise normally are. - - -Why not fix Pathlen=2?: - - The main reason I am not advocating that we always use 2 hops is that - in some situations, timing correlation evidence by itself may not be - considered as solid and convincing as an actual, uninterrupted, fully - traced path. Are these timing attacks as effective on a real network as - they are in simulation? Maybe the circuit multiplexing of Tor can serve - to frustrate them to a degree? Would an extralegal adversary or - authoritarian government even care? In the face of these situation - dependent unknowns, it should be up to the user to decide if this is - a concern for them or not. - - It should probably also be noted that even a false positive - rate of 1% for a 200k concurrent-user network could mean that for a - given node, a given stream could be confused with something like 10 - users, assuming ~200 nodes carry most of the traffic (ie 1000 users - each). Though of course to really know for sure, someone needs to do - an attack on a real network, unfortunately. - - Additionally, at some point cover traffic schemes may be implemented to - frustrate timing attacks on the first hop. It is possible some expert - users may do this ad-hoc already, and may wish to continue using 3 hops - for this reason. - - -Implementation: - - new_route_len() can be modified directly with a check of the - Pathlen option. However, circuit construction logic should be - altered so that both 2 hop and 3 hop users build the same types of - circuits, and the option should ultimately govern circuit selection, - not construction. This improves coverage against guard nodes being - able to passively profile users who aren't even using Tor. - PathlenCoinWeight, anyone? :) - - The exit policy hack is a bit more tricky. compare_addr_to_addr_policy - needs to return an alternate ADDR_POLICY_ACCEPTED_WILDCARD or - ADDR_POLICY_ACCEPTED_SPECIFIC return value for use in - circuit_is_acceptable. - - The leaky exit is trickier still.. handle_control_attachstream - does allow paths to exit at a given hop. Presumably something similar - can be done in connection_ap_handshake_process_socks, and elsewhere? - Circuit construction would also have to be performed such that the - 2nd hop's exit policy is what is considered, not the 3rd's. - - The entry_guard_t structure could have num_circ_failed and - num_circ_succeeded members such that if it exceeds F% circuit - extend failure rate to a second hop, it is removed from the entry list. - - F should be sufficiently high to avoid churn from normal Tor circuit - failure as determined by TorFlow scans. - - The Vidalia option should be presented as a radio button. - - -Migration: - - Phase 1: Adjust exit policy checks if Pathlen is set, implement leaky - circuit ability, and 2-3 hop circuit selection logic governed by - Pathlen. - - Phase 2: Experiment to determine the proper ratio of circuit - failures used to expire garbage or malicious guards via TorFlow - (pending Bug #440 backport+adoption). - - Phase 3: Implement guard expiration code to kick off failure-prone - guards and warn the user. Cap 2 hop guard duration to a proper number - of days determined sufficient to establish guard reliability (to be - determined by TorFlow). - - Phase 4: Make radiobutton in Vidalia, along with help entry - that explains in layman's terms the risks involved. - - Phase 5: Allow user to specify path length by HTTP URL suffix. - - -[1] http://p2pnet.net/story/11279 -[2] http://www.cs.umass.edu/~mwright/papers/levine-timing.pdf -[3] Proof available upon request ;) diff --git a/doc/spec/proposals/116-two-hop-paths-from-guard.txt b/doc/spec/proposals/116-two-hop-paths-from-guard.txt deleted file mode 100644 index 454b344ab..000000000 --- a/doc/spec/proposals/116-two-hop-paths-from-guard.txt +++ /dev/null @@ -1,120 +0,0 @@ -Filename: 116-two-hop-paths-from-guard.txt -Title: Two hop paths from entry guards -Version: $Revision$ -Last-Modified: $Date$ -Author: Michael Lieberman -Created: 26-Jun-2007 -Status: Dead - -This proposal is related to (but different from) Mike Perry's proposal 115 -"Two Hop Paths." - -Overview: - -Volunteers who run entry guards should have the option of using only 2 -additional tor nodes when constructing their own tor circuits. - -While the option of two hop paths should perhaps be extended to every client -(as discussed in Mike Perry's thread), I believe the anonymity properties of -two hop paths are particularly well-suited to client computers that are also -serving as entry guards. - -First I will describe the details of the strategy, as well as possible -avenues of attack. Then I will list advantages and disadvantages. Then, I -will discuss some possibly safer variations of the strategy, and finally -some implementation issues. - -Details: - -Suppose Alice is an entry guard, and wants to construct a two hop circuit. -Alice chooses a middle node at random (not using the entry guard strategy), -and gains anonymity by having her traffic look just like traffic from -someone else using her as an entry guard. - -Can Alice's middle node figure out that she is initiator of the traffic? I -can think of four possible approaches for distinguishing traffic from Alice -with traffic through Alice: - -1) Notice that communication from Alice comes too fast: Experimentation is -needed to determine if traffic from Alice can be distinguished from traffic -from a computer with a decent link to Alice. - -2) Monitor Alice's network traffic to discover the lack of incoming packets -at the appropriate times. If an adversary has this ability, then Alice -already has problems in the current system, because the adversary can run a -standard timing attack on Alice's traffic. - -3) Notice that traffic from Alice is unique in some way such that if Alice -was just one of 3 entry guards for this traffic, then the traffic should be -coming from two other entry guards as well. An example of "unique traffic" -could be always sending 117 packets every 3 minutes to an exit node that -exits to port 4661. However, if such patterns existed with sufficient -precision, then it seems to me that Tor already has a problem. (This "unique -traffic" may not be a problem if clients often end up choosing a single -entry guard because their other two are down. Does anyone know if this is -the case?) - -4) First, control the middle node *and* some other part of the traffic, -using standard attacks on a two hop circuit without entry nodes (my recent -paper on Browser-Based Attacks would work well for this -http://petworkshop.org/2007/papers/PET2007_preproc_Browser_based.pdf). With -control of the circuit, we can now cause "unique traffic" as in 3). -Alternatively, if we know something about Alice independently, and we can -see what websites are being visited, we might be able to guess that she is -the kind of person that would visit those websites. - -Anonymity Advantages: - --Alice never has the problem of choosing a malicious entry guard. In some -sense, Alice acts as her own entry guard. - -Anonymity Disadvantages: - --If Alice's traffic is identified as originating from herself (see above for -how hard that might be), then she has the anonymity of a 2 hop circuit -without entry guards. - -Additional advantages: - --A discussion of the latency advantages of two hop circuits is going on in -Mike Perry's thread already. --Also, we can advertise this change as "Run an entry guard and decrease your -own Tor latency." This incentive has the potential to add nodes to the -network, improving the network as a whole. - -Safer variations: - -To solve the "unique traffic" problem, Alice could use two hop paths only -1/3 of the time, and choose 2 other entry guards for the other 2/3 of the -time. All the advantages are now 1/3 as useful (possibly more, if the other -2 entry guards are not always up). - -To solve the problem that Alice's responses are too fast, Alice could delay -her responses (ideally based on some real data of response time when Alice -is used an entry guard). This loses most of the speed advantages of the two -hop path, but if Alice is a fast entry guard, it doesn't lose everything. It -also still has the (arguable) anonymity advantage that Alice doesn't have to -worry about having a malicious entry guard. - -Implementation details: -For Alice to remain anonymous using this strategy, she has to actually be -acting as an entry guard for other nodes. This means the two hop option can -only be available to whatever high-performance threshold is currently set on -entry guards. Alice may need to somehow check her own current status as an -entry guard before choosing this two hop strategy. - -Another thing to consider: suppose Alice is also an exit node. If the -fraction of exit nodes in existence is too small, she may rarely or never be -chosen as an entry guard. It would be sad if we offered an incentive to run -an entry guard that didn't extend to exit nodes. I suppose clients of Exit -nodes could pull the same trick, and bypass using Tor altogether (zero hop -paths), though that has additional issues.* - -Mike Lieberman -MIT - -*Why we shouldn't recommend Exit nodes pull the same trick: -1) Exit nodes would suffer heavily from the problem of "unique traffic" -mentioned above. -2) It would give governments an incentive to confiscate exit nodes to see if -they are pulling this trick. diff --git a/doc/spec/proposals/117-ipv6-exits.txt b/doc/spec/proposals/117-ipv6-exits.txt deleted file mode 100644 index c8402821e..000000000 --- a/doc/spec/proposals/117-ipv6-exits.txt +++ /dev/null @@ -1,412 +0,0 @@ -Filename: 117-ipv6-exits.txt -Title: IPv6 exits -Version: $Revision$ -Last-Modified: $Date$ -Author: coderman -Created: 10-Jul-2007 -Status: Accepted -Target: 0.2.1.x - -Overview - - Extend Tor for TCP exit via IPv6 transport and DNS resolution of IPv6 - addresses. This proposal does not imply any IPv6 support for OR - traffic, only exit and name resolution. - - -Contents - -0. Motivation - - As the IPv4 address space becomes more scarce there is increasing - effort to provide Internet services via the IPv6 protocol. Many - hosts are available at IPv6 endpoints which are currently - inaccessible for Tor users. - - Extending Tor to support IPv6 exit streams and IPv6 DNS name - resolution will allow users of the Tor network to access these hosts. - This capability would be present for those who do not currently have - IPv6 access, thus increasing the utility of Tor and furthering - adoption of IPv6. - - -1. Design - -1.1. General design overview - - There are three main components to this proposal. The first is a - method for routers to advertise their ability to exit IPv6 traffic. - The second is the manner in which routers resolve names to IPv6 - addresses. Last but not least is the method in which clients - communicate with Tor to resolve and connect to IPv6 endpoints - anonymously. - -1.2. Router IPv6 exit support - - In order to specify exit policies and IPv6 capability new directives - in the Tor configuration will be needed. If a router advertises IPv6 - exit policies in its descriptor this will signal the ability to - provide IPv6 exit. There are a number of additional default deny - rules associated with this new address space which are detailed in - the addendum. - - When Tor is started on a host it should check for the presence of a - global unicast IPv6 address and if present include the default IPv6 - exit policies and any user specified IPv6 exit policies. - - If a user provides IPv6 exit policies but no global unicast IPv6 - address is available Tor should generate a warning and not publish the - IPv6 policies in the router descriptor. - - It should be noted that IPv4 mapped IPv6 addresses are not valid exit - destinations. This mechanism is mainly used to interoperate with - both IPv4 and IPv6 clients on the same socket. Any attempts to use - an IPv4 mapped IPv6 address, perhaps to circumvent exit policy for - IPv4, must be refused. - -1.3. DNS name resolution of IPv6 addresses (AAAA records) - - In addition to exit support for IPv6 TCP connections, a method to - resolve domain names to their respective IPv6 addresses is also - needed. This is accomplished in the existing DNS system via AAAA - records. Routers will perform both A and AAAA requests when - resolving a name so that the client can utilize an IPv6 endpoint when - available or preferred. - - To avoid potential problems with caching DNS servers that behave - poorly all NXDOMAIN responses to AAAA requests should be ignored if a - successful response is received for an A request. This implies that - both AAAA and A requests will always be performed for each name - resolution. - - For reverse lookups on IPv6 addresses, like that used for - RESOLVE_PTR, Tor will perform the necessary PTR requests via - IP6.ARPA. - - All routers which perform DNS resolution on behalf of clients - (RELAY_RESOLVE) should perform and respond with both A and AAAA - resources. - - [NOTE: In a future version, when we extend the behavior of RESOLVE to - encapsulate more of real DNS, it will make sense to allow more - flexibility here. -nickm] - -1.4. Client interaction with IPv6 exit capability - -1.4.1. Usability goals - - There are a number of behaviors which Tor can provide when - interacting with clients that will improve the usability of IPv6 exit - capability. These behaviors are designed to make it simple for - clients to express a preference for IPv6 transport and utilize IPv6 - host services. - -1.4.2. SOCKSv5 IPv6 client behavior - - The SOCKS version 5 protocol supports IPv6 connections. When using - SOCKSv5 with hostnames it is difficult to determine if a client - wishes to use an IPv4 or IPv6 address to connect to the desired host - if it resolves to both address types. - - In order to make this more intuitive the SOCKSv5 protocol can be - supported on a local IPv6 endpoint, [::1] port 9050 for example. - When a client requests a connection to the desired host via an IPv6 - SOCKS connection Tor will prefer IPv6 addresses when resolving the - host name and connecting to the host. - - Likewise, RESOLVE and RESOLVE_PTR requests from an IPv6 SOCKS - connection will return IPv6 addresses when available, and fall back - to IPv4 addresses if not. - - [NOTE: This means that SocksListenAddress and DNSListenAddress should - support IPv6 addresses. Perhaps there should also be a general option - to have listeners that default to 127.0.0.1 and 0.0.0.0 listen - additionally or instead on ::1 and :: -nickm] - -1.4.3. MAPADDRESS behavior - - The MAPADDRESS capability supports clients that may not be able to - use the SOCKSv4a or SOCKSv5 hostname support to resolve names via - Tor. This ability should be extended to IPv6 addresses in SOCKSv5 as - well. - - When a client requests an address mapping from the wildcard IPv6 - address, [::0], the server will respond with a unique local IPv6 - address on success. It is important to note that there may be two - mappings for the same name if both an IPv4 and IPv6 address are - associated with the host. In this case a CONNECT to a mapped IPv6 - address should prefer IPv6 for the connection to the host, if - available, while CONNECT to a mapped IPv4 address will prefer IPv4. - - It should be noted that IPv6 does not provide the concept of a host - local subnet, like 127.0.0.0/8 in IPv4. For this reason integration - of Tor with IPv6 clients should consider a firewall or filter rule to - drop unique local addresses to or from the network when possible. - These packets should not be routed, however, keeping them off the - subnet entirely is worthwhile. - -1.4.3.1. Generating unique local IPv6 addresses - - The usual manner of generating a unique local IPv6 address is to - select a Global ID part randomly, along with a Subnet ID, and sharing - this prefix among the communicating parties who each have their own - distinct Interface ID. In this style a given Tor instance might - select a random Global and Subnet ID and provide MAPADDRESS - assignments with a random Interface ID as needed. This has the - potential to associate unique Global/Subnet identifiers with a given - Tor instance and may expose attacks against the anonymity of Tor - users. - - Tor avoid this potential problem entirely MAPADDRESS must always - generate the Global, Subnet, and Interface IDs randomly for each - request. It is also highly suggested that explicitly specifying an - IPv6 source address instead of the wildcard address not be supported - to ensure that a good random address is used. - -1.4.4. DNSProxy IPv6 client behavior - - A new capability in recent Tor versions is the transparent DNS proxy. - This feature will need to return both A and AAAA resource records - when responding to client name resolution requests. - - The transparent DNS proxy should also support reverse lookups for - IPv6 addresses. It is suggested that any such requests to the - deprecated IP6.INT domain should be translated to IP6.ARPA instead. - This translation is not likely to be used and is of low priority. - - It would be nice to support DNS over IPv6 transport as well, however, - this is not likely to be used and is of low priority. - -1.4.5. TransPort IPv6 client behavior - - Tor also provides transparent TCP proxy support via the Trans* - directives in the configuration. The TransListenAddress directive - should accept an IPv6 address in addition to IPv4 so that IPv6 TCP - connections can be transparently proxied. - -1.5. Additional changes - - The RedirectExit option should be deprecated rather than extending - this feature to IPv6. - - -2. Spec changes - -2.1. Tor specification - - In '6.2. Opening streams and transferring data' the following should - be changed to indicate IPv6 exit capability: - - "No version of Tor currently generates the IPv6 format." - - In '6.4. Remote hostname lookup' the following should be updated to - reflect use of ip6.arpa in addition to in-addr.arpa. - - "For a reverse lookup, the OP sends a RELAY_RESOLVE cell containing an - in-addr.arpa address." - - In 'A.1. Differences between spec and implementation' the following - should be updated to indicate IPv6 exit capability: - - "The current codebase has no IPv6 support at all." - - [NOTE: the EXITPOLICY end-cell reason says that it can hold an ipv4 or an - ipv6 address, but doesn't say how. We may want a separate EXITPOLICY2 - type that can hold an ipv6 address, since the way we encode ipv6 - addresses elsewhere ("0.0.0.0 indicates that the next 16 bytes are ipv6") - is a bit dumb. -nickm] - [Actually, the length field lets us distinguish EXITPOLICY. -nickm] - -2.2. Directory specification - - In '2.1. Router descriptor format' a new set of directives is needed - for IPv6 exit policy. The existing accept/reject directives should - be clarified to indicate IPv4 or wildcard address relevance. The new - IPv6 directives will be in the form of: - - "accept6" exitpattern NL - "reject6" exitpattern NL - - The section describing accept6/reject6 should explain that the - presence of accept6 or reject6 exit policies in a router descriptor - signals the ability of that router to exit IPv6 traffic (according to - IPv6 exit policies). - - The "[::]/0" notation is used to represent "all IPv6 addresses". - "[::0]/0" may also be used for this representation. - - If a user specifies a 'reject6 [::]/0:*' policy in the Tor - configuration this will be interpreted as forcing no IPv6 exit - support and no accept6/reject6 policies will be included in the - published descriptor. This will prevent IPv6 exit if the router host - has a global unicast IPv6 address present. - - It is important to note that a wildcard address in an accept or - reject policy applies to both IPv4 and IPv6 addresses. - -2.3. Control specification - - In '3.8. MAPADDRESS' the potential to have to addresses for a given - name should be explained. The method for generating unique local - addresses for IPv6 mappings needs explanation as described above. - - When IPv6 addresses are used in this document they should include the - brackets for consistency. For example, the null IPv6 address should - be written as "[::0]" and not "::0". The control commands will - expect the same syntax as well. - - In '3.9. GETINFO' the "address" command should return both public - IPv4 and IPv6 addresses if present. These addresses should be - separated via \r\n. - - -2.4. Tor SOCKS extensions - - In '2. Name lookup' a description of IPv6 address resolution is - needed for SOCKSv5 as described above. IPv6 addresses should be - supported in both the RESOLVE and RESOLVE_PTR extensions. - - A new section describing the ability to accept SOCKSv5 clients on a - local IPv6 address to indicate a preference for IPv6 transport as - described above is also needed. The behavior of Tor SOCKSv5 proxy - with an IPv6 preference should be explained, for example, preferring - IPv6 transport to a named host with both IPv4 and IPv6 addresses - available (A and AAAA records). - - -3. Questions and concerns - -3.1. DNS A6 records - - A6 is explicitly avoided in this document. There are potential - reasons for implementing this, however, the inherent complexity of - the protocol and resolvers make this unappealing. Is there a - compelling reason to consider A6 as part of IPv6 exit support? - - [IMO not till anybody needs it. -nickm] - -3.2. IPv4 and IPv6 preference - - The design above tries to infer a preference for IPv4 or IPv6 - transport based on client interactions with Tor. It might be useful - to provide more explicit control over this preference. For example, - an IPv4 SOCKSv5 client may want to use IPv6 transport to named hosts - in CONNECT requests while the current implementation would assume an - IPv4 preference. Should more explicit control be available, through - either configuration directives or control commands? - - Many applications support a inet6-only or prefer-family type option - that provides the user manual control over address preference. This - could be provided as a Tor configuration option. - - An explicit preference is still possible by resolving names and then - CONNECTing to an IPv4 or IPv6 address as desired, however, not all - client applications may have this option available. - -3.3. Support for IPv6 only transparent proxy clients - - It may be useful to support IPv6 only transparent proxy clients using - IPv4 mapped IPv6 like addresses. This would require transparent DNS - proxy using IPv6 transport and the ability to map A record responses - into IPv4 mapped IPv6 like addresses in the manner described in the - "NAT-PT" RFC for a traditional Basic-NAT-PT with DNS-ALG. The - transparent TCP proxy would thus need to detect these mapped addresses - and connect to the desired IPv4 host. - - The IPv6 prefix used for this purpose must not be the actual IPv4 - mapped IPv6 address prefix, though the manner in which IPv4 addresses - are embedded in IPv6 addresses would be the same. - - The lack of any IPv6 only hosts which would use this transparent proxy - method makes this a lot of work for very little gain. Is there a - compelling reason to support this NAT-PT like capability? - -3.4. IPv6 DNS and older Tor routers - - It is expected that many routers will continue to run with older - versions of Tor when the IPv6 exit capability is released. Clients - who wish to use IPv6 will need to route RELAY_RESOLVE requests to the - newer routers which will respond with both A and AAAA resource - records when possible. - - One way to do this is to route RELAY_RESOLVE requests to routers with - IPv6 exit policies published, however, this would not utilize current - routers that can resolve IPv6 addresses even if they can't exit such - traffic. - - There was also concern expressed about the ability of existing clients - to cope with new RELAY_RESOLVE responses that contain IPv6 addresses. - If this breaks backward compatibility, a new request type may be - necessary, like RELAY_RESOLVE6, or some other mechanism of indicating - the ability to parse IPv6 responses when making the request. - -3.5. IPv4 and IPv6 bindings in MAPADDRESS - - It may be troublesome to try and support two distinct address mappings - for the same name in the existing MAPADDRESS implementation. If this - cannot be accommodated then the behavior should replace existing - mappings with the new address regardless of family. A warning when - this occurs would be useful to assist clients who encounter problems - when both an IPv4 and IPv6 application are using MAPADDRESS for the - same names concurrently, causing lost connections for one of them. - -4. Addendum - -4.1. Sample IPv6 default exit policy - - reject 0.0.0.0/8 - reject 169.254.0.0/16 - reject 127.0.0.0/8 - reject 192.168.0.0/16 - reject 10.0.0.0/8 - reject 172.16.0.0/12 - reject6 [0000::]/8 - reject6 [0100::]/8 - reject6 [0200::]/7 - reject6 [0400::]/6 - reject6 [0800::]/5 - reject6 [1000::]/4 - reject6 [4000::]/3 - reject6 [6000::]/3 - reject6 [8000::]/3 - reject6 [A000::]/3 - reject6 [C000::]/3 - reject6 [E000::]/4 - reject6 [F000::]/5 - reject6 [F800::]/6 - reject6 [FC00::]/7 - reject6 [FE00::]/9 - reject6 [FE80::]/10 - reject6 [FEC0::]/10 - reject6 [FF00::]/8 - reject *:25 - reject *:119 - reject *:135-139 - reject *:445 - reject *:1214 - reject *:4661-4666 - reject *:6346-6429 - reject *:6699 - reject *:6881-6999 - accept *:* - # accept6 [2000::]/3:* is implied - -4.2. Additional resources - - 'DNS Extensions to Support IP Version 6' - http://www.ietf.org/rfc/rfc3596.txt - - 'DNS Extensions to Support IPv6 Address Aggregation and Renumbering' - http://www.ietf.org/rfc/rfc2874.txt - - 'SOCKS Protocol Version 5' - http://www.ietf.org/rfc/rfc1928.txt - - 'Unique Local IPv6 Unicast Addresses' - http://www.ietf.org/rfc/rfc4193.txt - - 'INTERNET PROTOCOL VERSION 6 ADDRESS SPACE' - http://www.iana.org/assignments/ipv6-address-space - - 'Network Address Translation - Protocol Translation (NAT-PT)' - http://www.ietf.org/rfc/rfc2766.txt diff --git a/doc/spec/proposals/118-multiple-orports.txt b/doc/spec/proposals/118-multiple-orports.txt deleted file mode 100644 index 1bef2504d..000000000 --- a/doc/spec/proposals/118-multiple-orports.txt +++ /dev/null @@ -1,86 +0,0 @@ -Filename: 118-multiple-orports.txt -Title: Advertising multiple ORPorts at once -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 09-Jul-2007 -Status: Accepted -Target: 0.2.1.x - -Overview: - - This document is a proposal for servers to advertise multiple - address/port combinations for their ORPort. - -Motivation: - - Sometimes servers want to support multiple ports for incoming - connections, either in order to support multiple address families, to - better use multiple interfaces, or to support a variety of - FascistFirewallPorts settings. This is easy to set up now, but - there's no way to advertise it to clients. - -New descriptor syntax: - - We add a new line in the router descriptor, "or-address". This line - can occur zero, one, or multiple times. Its format is: - - or-address SP ADDRESS ":" PORTLIST NL - - ADDRESS = IP6ADDR / IP4ADDR - IPV6ADDR = an ipv6 address, surrounded by square brackets. - IPV4ADDR = an ipv4 address, represented as a dotted quad. - PORTLIST = PORTSPEC | PORTSPEC "," PORTLIST - PORTSPEC = PORT | PORT "-" PORT - - [This is the regular format for specifying sets of addresses and - ports in Tor.] - -New OR behavior: - - We add two more options to supplement ORListenAddress: - ORPublishedListenAddress, and ORPublishAddressSet. The former - listens on an address-port combination and publishes it in addition - to the regular address. The latter advertises a set of address-port - combinations, but does not listen on them. [To use this option, the - server operator should set up port forwarding to the regular ORPort, - as for example with firewall rules.] - - Servers should extend their testing to include advertised addresses - and ports. No address or port should be advertised until it's been - tested. [This might get expensive in practice.] - -New authority behavior: - - Authorities should spot-test descriptors, and reject any where a - substantial part of the addresses can't be reached. - -New client behavior: - - When connecting to another server, clients SHOULD pick an - address-port ocmbination at random as supported by their - reachableaddresses. If a client has a connection to a server at one - address, it SHOULD use that address for any simultaneous connections - to that server. Clients SHOULD use the canonical address for any - server when generating extend cells. - -Not addressed here: - - * There's no reason to listen on multiple dirports; current Tors - mostly don't connect directly to the dirport anyway. - - * It could be advantageous to list something about extra addresses in - the network-status document. This would, however, eat space there. - More analysis is needed, particularly in light of proposal 141 - ("Download server descriptors on demand") - -Dependencies: - - Testing for canonical connections needs to be implemented before it's - safe to use this proposal. - - -Notes 3 July: - - Write up the simple version of this. No ranges needed yet. No - networkstatus chagnes yet. - diff --git a/doc/spec/proposals/119-controlport-auth.txt b/doc/spec/proposals/119-controlport-auth.txt deleted file mode 100644 index dc57a2736..000000000 --- a/doc/spec/proposals/119-controlport-auth.txt +++ /dev/null @@ -1,142 +0,0 @@ -Filename: 119-controlport-auth.txt -Title: New PROTOCOLINFO command for controllers -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 14-Aug-2007 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - Here we describe how to help controllers locate the cookie - authentication file when authenticating to Tor, so we can a) require - authentication by default for Tor controllers and b) still keep - things usable. Also, we propose an extensible, general-purpose mechanism - for controllers to learn about a Tor instance's protocol and - authentication requirements before authenticating. - -The Problem: - - When we first added the controller protocol, we wanted to make it - easy for people to play with it, so by default we didn't require any - authentication from controller programs. We allowed requests only from - localhost as a stopgap measure for security. - - Due to an increasing number of vulnerabilities based on this approach, - it's time to add authentication in default configurations. - - We have a number of goals: - - We want the default Vidalia bundles to transparently work. That - means we don't want the users to have to type in or know a password. - - We want to allow multiple controller applications to connect to the - control port. So if Vidalia is launching Tor, it can't just keep the - secrets to itself. - - Right now there are three authentication approaches supported - by the control protocol: NULL, CookieAuthentication, and - HashedControlPassword. See Sec 5.1 in control-spec.txt for details. - - There are a couple of challenges here. The first is: if the controller - launches Tor, how should we teach Tor what authentication approach - it should require, and the secret that goes along with it? Next is: - how should this work when the controller attaches to an existing Tor, - rather than launching Tor itself? - - Cookie authentication seems most amenable to letting multiple controller - applications interact with Tor. But that brings in yet another question: - how does the controller guess where to look for the cookie file, - without first knowing what DataDirectory Tor is using? - -Design: - - We should add a new controller command PROTOCOLINFO that can be sent - as a valid first command (the others being AUTHENTICATE and QUIT). If - PROTOCOLINFO is sent as the first command, the second command must be - either a successful AUTHENTICATE or a QUIT. - - If the initial command sequence is not valid, Tor closes the connection. - - -Spec: - - C: "PROTOCOLINFO" *(SP PIVERSION) CRLF - S: "250+PROTOCOLINFO" SP PIVERSION CRLF *InfoLine "250 OK" CRLF - - InfoLine = AuthLine / VersionLine / OtherLine - - AuthLine = "250-AUTH" SP "METHODS=" AuthMethod *(",")AuthMethod - *(SP "COOKIEFILE=" AuthCookieFile) CRLF - VersionLine = "250-VERSION" SP "Tor=" TorVersion [SP Arguments] CRLF - - AuthMethod = - "NULL" / ; No authentication is required - "HASHEDPASSWORD" / ; A controller must supply the original password - "COOKIE" / ; A controller must supply the contents of a cookie - - AuthCookieFile = QuotedString - TorVersion = QuotedString - - OtherLine = "250-" Keyword [SP Arguments] CRLF - - For example: - - C: PROTOCOLINFO CRLF - S: "250+PROTOCOLINFO 1" CRLF - S: "250-AUTH Methods=HASHEDPASSWORD,COOKIE COOKIEFILE="/tor/cookie"" CRLF - S: "250-VERSION Tor=0.2.0.5-alpha" CRLF - S: "250 OK" CRLF - - Tor MAY give its InfoLines in any order; controllers MUST ignore InfoLines - with keywords it does not recognize. Controllers MUST ignore extraneous - data on any InfoLine. - - PIVERSION is there in case we drastically change the syntax one day. For - now it should always be "1", for the controller protocol. Controllers MAY - provide a list of the protocol versions they support; Tor MAY select a - version that the controller does not support. - - Right now only two "topics" (AUTH and VERSION) are included, but more - may be included in the future. Controllers must accept lines with - unexpected topics. - - AuthCookieFile = QuotedString - - AuthMethod is used to specify one or more control authentication - methods that Tor currently accepts. - - AuthCookieFile specifies the absolute path and filename of the - authentication cookie that Tor is expecting and is provided iff - the METHODS field contains the method "COOKIE". Controllers MUST handle - escape sequences inside this string. - - The VERSION line contains the Tor version. - - [What else might we want to include that could be useful? -RD] - -Compatibility: - - Tor 0.1.2.16 and 0.2.0.4-alpha hang up after the first failed - command. Earlier Tors don't know about this command but don't hang - up. That means controllers will need a mechanism for distinguishing - whether they're talking to a Tor that speaks PROTOCOLINFO or not. - - I suggest that the controllers attempt a PROTOCOLINFO. Then: - - If it works, great. Authenticate as required. - - If they get hung up on, reconnect and do a NULL AUTHENTICATE. - - If it's unrecognized but they're not hung up on, do a NULL - AUTHENTICATE. - -Unsolved problems: - - If Torbutton wants to be a Tor controller one day... talking TCP is - bad enough, but reading from the filesystem is even harder. Is there - a way to let simple programs work with the controller port without - needing all the auth infrastructure? - - Once we put this approach in place, the next vulnerability we see will - involve an attacker somehow getting read access to the victim's files - --- and then we're back where we started. This means we still need - to think about how to demand password-based authentication without - bothering the user about it. - diff --git a/doc/spec/proposals/120-shutdown-descriptors.txt b/doc/spec/proposals/120-shutdown-descriptors.txt deleted file mode 100644 index dc1265b03..000000000 --- a/doc/spec/proposals/120-shutdown-descriptors.txt +++ /dev/null @@ -1,85 +0,0 @@ -Filename: 120-shutdown-descriptors.txt -Title: Shutdown descriptors when Tor servers stop -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 15-Aug-2007 -Status: Dead - -[Proposal dead as of 11 Jul 2008. The point of this proposal was to give -routers a good way to get out of the networkstatus early, but proposal -138 (already implemented) has achieved this.] - -Overview: - - Tor servers should publish a last descriptor whenever they shut down, - to let others know that they are no longer offering service. - -The Problem: - - The main reason for this is in reaction to Internet services that want - to treat connections from the Tor network differently. Right now, - if a user experiments with turning on the "relay" functionality, he - is punished by being locked out of some websites, some IRC networks, - etc --- and this lockout persists for several days even after he turns - the server off. - -Design: - - During the "slow shutdown" period if exiting, or shortly after the - user sets his ORPort back to 0 if not exiting, Tor should publish a - final descriptor with the following characteristics: - - 1) Exit policy is listed as "reject *:*" - 2) It includes a new entry called "opt shutdown 1" - - The first step is so current blacklists will no longer list this node - as exiting to whatever the service is. - - The second step is so directory authorities can avoid wasting time - doing reachability testing. Authorities should automatically not list - as Running any router whose latest descriptor says it shut down. - - [I originally had in mind a third step --- Advertised bandwidth capacity - is listed as "0" --- so current Tor clients will skip over this node - when building most circuits. But since clients won't fetch descriptors - from nodes not listed as Running, this step seems pointless. -RD] - -Spec: - - TBD but should be pretty straightforward. - -Security issues: - - Now external people can learn exactly when a node stopped offering - relay service. How bad is this? I can see a few minor attacks based - on this knowledge, but on the other hand as it is we don't really take - any steps to keep this information secret. - -Overhead issues: - - We are creating more descriptors that want to be remembered. However, - since the router won't be marked as Running, ordinary clients won't - fetch the shutdown descriptors. Caches will, though. I hope this is ok. - -Implementation: - - To make things easy, we should publish the shutdown descriptor only - on controlled shutdown (SIGINT as opposed to SIGTERM). That would - leave enough time for publishing that we probably wouldn't need any - extra synchronization code. - - If that turns out to be too unintuitive for users, I could imagine doing - it on SIGTERMs too, and just delaying exit until we had successfully - published to at least one authority, at which point we'd hope that it - propagated from there. - -Acknowledgements: - - tup suggested this idea. - -Comments: - - 2) Maybe add a rule "Don't do this for hibernation if we expect to wake - up before the next consensus is published"? - - NM 9 Oct 2007 diff --git a/doc/spec/proposals/121-hidden-service-authentication.txt b/doc/spec/proposals/121-hidden-service-authentication.txt deleted file mode 100644 index 828bf3c92..000000000 --- a/doc/spec/proposals/121-hidden-service-authentication.txt +++ /dev/null @@ -1,778 +0,0 @@ -Filename: 121-hidden-service-authentication.txt -Title: Hidden Service Authentication -Version: $Revision$ -Last-Modified: $Date$ -Author: Tobias Kamm, Thomas Lauterbach, Karsten Loesing, Ferdinand Rieger, - Christoph Weingarten -Created: 10-Sep-2007 -Status: Finished -Implemented-In: 0.2.1.x - -Change history: - - 26-Sep-2007 Initial proposal for or-dev - 08-Dec-2007 Incorporated comments by Nick posted to or-dev on 10-Oct-2007 - 15-Dec-2007 Rewrote complete proposal for better readability, modified - authentication protocol, merged in personal notes - 24-Dec-2007 Replaced misleading term "authentication" by "authorization" - and added some clarifications (comments by Sven Kaffille) - 28-Apr-2008 Updated most parts of the concrete authorization protocol - 04-Jul-2008 Add a simple algorithm to delay descriptor publication for - different clients of a hidden service - 19-Jul-2008 Added INTRODUCE1V cell type (1.2), improved replay - protection for INTRODUCE2 cells (1.3), described limitations - for auth protocols (1.6), improved hidden service protocol - without client authorization (2.1), added second, more - scalable authorization protocol (2.2), rewrote existing - authorization protocol (2.3); changes based on discussion - with Nick - 31-Jul-2008 Limit maximum descriptor size to 20 kilobytes to prevent - abuse. - 01-Aug-2008 Use first part of Diffie-Hellman handshake for replay - protection instead of rendezvous cookie. - 01-Aug-2008 Remove improved hidden service protocol without client - authorization (2.1). It might get implemented in proposal - 142. - -Overview: - - This proposal deals with a general infrastructure for performing - authorization (not necessarily implying authentication) of requests to - hidden services at three points: (1) when downloading and decrypting - parts of the hidden service descriptor, (2) at the introduction point, - and (3) at Bob's Tor client before contacting the rendezvous point. A - service provider will be able to restrict access to his service at these - three points to authorized clients only. Further, the proposal contains - specific authorization protocols as instances that implement the - presented authorization infrastructure. - - This proposal is based on v2 hidden service descriptors as described in - proposal 114 and introduced in version 0.2.0.10-alpha. - - The proposal is structured as follows: The next section motivates the - integration of authorization mechanisms in the hidden service protocol. - Then we describe a general infrastructure for authorization in hidden - services, followed by specific authorization protocols for this - infrastructure. At the end we discuss a number of attacks and non-attacks - as well as compatibility issues. - -Motivation: - - The major part of hidden services does not require client authorization - now and won't do so in the future. To the contrary, many clients would - not want to be (pseudonymously) identifiable by the service (though this - is unavoidable to some extent), but rather use the service - anonymously. These services are not addressed by this proposal. - - However, there may be certain services which are intended to be accessed - by a limited set of clients only. A possible application might be a - wiki or forum that should only be accessible for a closed user group. - Another, less intuitive example might be a real-time communication - service, where someone provides a presence and messaging service only to - his buddies. Finally, a possible application would be a personal home - server that should be remotely accessed by its owner. - - Performing authorization for a hidden service within the Tor network, as - proposed here, offers a range of advantages compared to allowing all - client connections in the first instance and deferring authorization to - the transported protocol: - - (1) Reduced traffic: Unauthorized requests would be rejected as early as - possible, thereby reducing the overall traffic in the network generated - by establishing circuits and sending cells. - - (2) Better protection of service location: Unauthorized clients could not - force Bob to create circuits to their rendezvous points, thus preventing - the attack described by Øverlier and Syverson in their paper "Locating - Hidden Servers" even without the need for guards. - - (3) Hiding activity: Apart from performing the actual authorization, a - service provider could also hide the mere presence of his service from - unauthorized clients when not providing hidden service descriptors to - them, rejecting unauthorized requests already at the introduction - point (ideally without leaking presence information at any of these - points), or not answering unauthorized introduction requests. - - (4) Better protection of introduction points: When providing hidden - service descriptors to authorized clients only and encrypting the - introduction points as described in proposal 114, the introduction points - would be unknown to unauthorized clients and thereby protected from DoS - attacks. - - (5) Protocol independence: Authorization could be performed for all - transported protocols, regardless of their own capabilities to do so. - - (6) Ease of administration: A service provider running multiple hidden - services would be able to configure access at a single place uniformly - instead of doing so for all services separately. - - (7) Optional QoS support: Bob could adapt his node selection algorithm - for building the circuit to Alice's rendezvous point depending on a - previously guaranteed QoS level, thus providing better latency or - bandwidth for selected clients. - - A disadvantage of performing authorization within the Tor network is - that a hidden service cannot make use of authorization data in - the transported protocol. Tor hidden services were designed to be - independent of the transported protocol. Therefore it's only possible to - either grant or deny access to the whole service, but not to specific - resources of the service. - - Authorization often implies authentication, i.e. proving one's identity. - However, when performing authorization within the Tor network, untrusted - points should not gain any useful information about the identities of - communicating parties, neither server nor client. A crucial challenge is - to remain anonymous towards directory servers and introduction points. - However, trying to hide identity from the hidden service is a futile - task, because a client would never know if he is the only authorized - client and therefore perfectly identifiable. Therefore, hiding client - identity from the hidden service is not an aim of this proposal. - - The current implementation of hidden services does not provide any kind - of authorization. The hidden service descriptor version 2, introduced by - proposal 114, was designed to use a descriptor cookie for downloading and - decrypting parts of the descriptor content, but this feature is not yet - in use. Further, most relevant cell formats specified in rend-spec - contain fields for authorization data, but those fields are neither - implemented nor do they suffice entirely. - -Details: - - 1. General infrastructure for authorization to hidden services - - We spotted three possible authorization points in the hidden service - protocol: - - (1) when downloading and decrypting parts of the hidden service - descriptor, - (2) at the introduction point, and - (3) at Bob's Tor client before contacting the rendezvous point. - - The general idea of this proposal is to allow service providers to - restrict access to some or all of these points to authorized clients - only. - - 1.1. Client authorization at directory - - Since the implementation of proposal 114 it is possible to combine a - hidden service descriptor with a so-called descriptor cookie. If done so, - the descriptor cookie becomes part of the descriptor ID, thus having an - effect on the storage location of the descriptor. Someone who has learned - about a service, but is not aware of the descriptor cookie, won't be able - to determine the descriptor ID and download the current hidden service - descriptor; he won't even know whether the service has uploaded a - descriptor recently. Descriptor IDs are calculated as follows (see - section 1.2 of rend-spec for the complete specification of v2 hidden - service descriptors): - - descriptor-id = - H(service-id | H(time-period | descriptor-cookie | replica)) - - Currently, service-id is equivalent to permanent-id which is calculated - as in the following formula. But in principle it could be any public - key. - - permanent-id = H(permanent-key)[:10] - - The second purpose of the descriptor cookie is to encrypt the list of - introduction points, including optional authorization data. Hence, the - hidden service directories won't learn any introduction information from - storing a hidden service descriptor. This feature is implemented but - unused at the moment. So this proposal will harness the advantages - of proposal 114. - - The descriptor cookie can be used for authorization by keeping it secret - from everyone but authorized clients. A service could then decide whether - to publish hidden service descriptors using that descriptor cookie later - on. An authorized client being aware of the descriptor cookie would be - able to download and decrypt the hidden service descriptor. - - The number of concurrently used descriptor cookies for one hidden service - is not restricted. A service could use a single descriptor cookie for all - users, a distinct cookie per user, or something in between, like one - cookie per group of users. It is up to the specific protocol and how it - is applied by a service provider. - - Two or more hidden service descriptors for different groups or users - should not be uploaded at the same time. A directory node could conclude - easily that the descriptors were issued by the same hidden service, thus - being able to link the two groups or users. Therefore, descriptors for - different users or clients that ought to be stored on the same directory - are delayed, so that only one descriptor is uploaded to a directory at a - time. The remaining descriptors are uploaded with a delay of up to - 30 seconds. - Further, descriptors for different groups or users that are to be stored - on different directories are delayed for a random time of up to 30 - seconds to hide relations from colluding directories. Certainly, this - does not prevent linking entirely, but it makes it somewhat harder. - There is a conflict between hiding links between clients and making a - service available in a timely manner. - - Although this part of the proposal is meant to describe a general - infrastructure for authorization, changing the way of using the - descriptor cookie to look up hidden service descriptors, e.g. applying - some sort of asymmetric crypto system, would require in-depth changes - that would be incompatible to v2 hidden service descriptors. On the - contrary, using another key for en-/decrypting the introduction point - part of a hidden service descriptor, e.g. a different symmetric key or - asymmetric encryption, would be easy to implement and compatible to v2 - hidden service descriptors as understood by hidden service directories - (clients and services would have to be upgraded anyway for using the new - features). - - An adversary could try to abuse the fact that introduction points can be - encrypted by storing arbitrary, unrelated data in the hidden service - directory. This abuse can be limited by setting a hard descriptor size - limit, forcing the adversary to split data into multiple chunks. There - are some limitations that make splitting data across multiple descriptors - unattractive: 1) The adversary would not be able to choose descriptor IDs - freely and would therefore have to implement his own indexing - structure. 2) Validity of descriptors is limited to at most 24 hours - after which descriptors need to be republished. - - The regular descriptor size in bytes is 745 + num_ipos * 837 + auth_data. - A large descriptor with 7 introduction points and 5 kilobytes of - authorization data would be 11724 bytes in size. The upper size limit of - descriptors should be set to 20 kilobytes, which limits the effect of - abuse while retaining enough flexibility in designing authorization - protocols. - - 1.2. Client authorization at introduction point - - The next possible authorization point after downloading and decrypting - a hidden service descriptor is the introduction point. It may be important - for authorization, because it bears the last chance of hiding presence - of a hidden service from unauthorized clients. Further, performing - authorization at the introduction point might reduce traffic in the - network, because unauthorized requests would not be passed to the - hidden service. This applies to those clients who are aware of a - descriptor cookie and thereby of the hidden service descriptor, but do - not have authorization data to pass the introduction point or access the - service (such a situation might occur when authorization data for - authorization at the directory is not issued on a per-user basis, but - authorization data for authorization at the introduction point is). - - It is important to note that the introduction point must be considered - untrustworthy, and therefore cannot replace authorization at the hidden - service itself. Nor should the introduction point learn any sensitive - identifiable information from either the service or the client. - - In order to perform authorization at the introduction point, three - message formats need to be modified: (1) v2 hidden service descriptors, - (2) ESTABLISH_INTRO cells, and (3) INTRODUCE1 cells. - - A v2 hidden service descriptor needs to contain authorization data that - is introduction-point-specific and sometimes also authorization data - that is introduction-point-independent. Therefore, v2 hidden service - descriptors as specified in section 1.2 of rend-spec already contain two - reserved fields "intro-authorization" and "service-authorization" - (originally, the names of these fields were "...-authentication") - containing an authorization type number and arbitrary authorization - data. We propose that authorization data consists of base64 encoded - objects of arbitrary length, surrounded by "-----BEGIN MESSAGE-----" and - "-----END MESSAGE-----". This will increase the size of hidden service - descriptors, but this is allowed since there is no strict upper limit. - - The current ESTABLISH_INTRO cells as described in section 1.3 of - rend-spec do not contain either authorization data or version - information. Therefore, we propose a new version 1 of the ESTABLISH_INTRO - cells adding these two issues as follows: - - V Format byte: set to 255 [1 octet] - V Version byte: set to 1 [1 octet] - KL Key length [2 octets] - PK Bob's public key [KL octets] - HS Hash of session info [20 octets] - AUTHT The auth type that is supported [1 octet] - AUTHL Length of auth data [2 octets] - AUTHD Auth data [variable] - SIG Signature of above information [variable] - - From the format it is possible to determine the maximum allowed size for - authorization data: given the fact that cells are 512 octets long, of - which 498 octets are usable (see section 6.1 of tor-spec), and assuming - 1024 bit = 128 octet long keys, there are 215 octets left for - authorization data. Hence, authorization protocols are bound to use no - more than these 215 octets, regardless of the number of clients that - shall be authenticated at the introduction point. Otherwise, one would - need to send multiple ESTABLISH_INTRO cells or split them up, which we do - not specify here. - - In order to understand a v1 ESTABLISH_INTRO cell, the implementation of - a relay must have a certain Tor version. Hidden services need to be able - to distinguish relays being capable of understanding the new v1 cell - formats and perform authorization. We propose to use the version number - that is contained in networkstatus documents to find capable - introduction points. - - The current INTRODUCE1 cell as described in section 1.8 of rend-spec is - not designed to carry authorization data and has no version number, too. - Unfortunately, unversioned INTRODUCE1 cells consist only of a fixed-size, - seemingly random PK_ID, followed by the encrypted INTRODUCE2 cell. This - makes it impossible to distinguish unversioned INTRODUCE1 cells from any - later format. In particular, it is not possible to introduce some kind of - format and version byte for newer versions of this cell. That's probably - where the comment "[XXX011 want to put intro-level auth info here, but no - version. crap. -RD]" that was part of rend-spec some time ago comes from. - - We propose that new versioned INTRODUCE1 cells use the new cell type 41 - RELAY_INTRODUCE1V (where V stands for versioned): - - Cleartext - V Version byte: set to 1 [1 octet] - PK_ID Identifier for Bob's PK [20 octets] - AUTHT The auth type that is included [1 octet] - AUTHL Length of auth data [2 octets] - AUTHD Auth data [variable] - Encrypted to Bob's PK: - (RELAY_INTRODUCE2 cell) - - The maximum length of contained authorization data depends on the length - of the contained INTRODUCE2 cell. A calculation follows below when - describing the INTRODUCE2 cell format we propose to use. - - 1.3. Client authorization at hidden service - - The time when a hidden service receives an INTRODUCE2 cell constitutes - the last possible authorization point during the hidden service - protocol. Performing authorization here is easier than at the other two - authorization points, because there are no possibly untrusted entities - involved. - - In general, a client that is successfully authorized at the introduction - point should be granted access at the hidden service, too. Otherwise, the - client would receive a positive INTRODUCE_ACK cell from the introduction - point and conclude that it may connect to the service, but the request - will be dropped without notice. This would appear as a failure to - clients. Therefore, the number of cases in which a client successfully - passes the introduction point but fails at the hidden service should be - zero. However, this does not lead to the conclusion that the - authorization data used at the introduction point and the hidden service - must be the same, but only that both authorization data should lead to - the same authorization result. - - Authorization data is transmitted from client to server via an - INTRODUCE2 cell that is forwarded by the introduction point. There are - versions 0 to 2 specified in section 1.8 of rend-spec, but none of these - contain fields for carrying authorization data. We propose a slightly - modified version of v3 INTRODUCE2 cells that is specified in section - 1.8.1 and which is not implemented as of December 2007. In contrast to - the specified v3 we avoid specifying (and implementing) IPv6 capabilities, - because Tor relays will be required to support IPv4 addresses for a long - time in the future, so that this seems unnecessary at the moment. The - proposed format of v3 INTRODUCE2 cells is as follows: - - VER Version byte: set to 3. [1 octet] - AUTHT The auth type that is used [1 octet] - AUTHL Length of auth data [2 octets] - AUTHD Auth data [variable] - TS Timestamp (seconds since 1-1-1970) [4 octets] - IP Rendezvous point's address [4 octets] - PORT Rendezvous point's OR port [2 octets] - ID Rendezvous point identity ID [20 octets] - KLEN Length of onion key [2 octets] - KEY Rendezvous point onion key [KLEN octets] - RC Rendezvous cookie [20 octets] - g^x Diffie-Hellman data, part 1 [128 octets] - - The maximum possible length of authorization data is related to the - enclosing INTRODUCE1V cell. A v3 INTRODUCE2 cell with - 1024 bit = 128 octets long public key without any authorization data - occupies 306 octets (AUTHL is only used when AUTHT has a value != 0), - plus 58 octets for hybrid public key encryption (see - section 5.1 of tor-spec on hybrid encryption of CREATE cells). The - surrounding INTRODUCE1V cell requires 24 octets. This leaves only 110 - of the 498 available octets free, which must be shared between - authorization data to the introduction point _and_ to the hidden - service. - - When receiving a v3 INTRODUCE2 cell, Bob checks whether a client has - provided valid authorization data to him. He also requires that the - timestamp is no more than 30 minutes in the past or future and that the - first part of the Diffie-Hellman handshake has not been used in the past - 60 minutes to prevent replay attacks by rogue introduction points. (The - reason for not using the rendezvous cookie to detect replays---even - though it is only sent once in the current design---is that it might be - desirable to re-use rendezvous cookies for multiple introduction requests - in the future.) If all checks pass, Bob builds a circuit to the provided - rendezvous point. Otherwise he drops the cell. - - 1.4. Summary of authorization data fields - - In summary, the proposed descriptor format and cell formats provide the - following fields for carrying authorization data: - - (1) The v2 hidden service descriptor contains: - - a descriptor cookie that is used for the lookup process, and - - an arbitrary encryption schema to ensure authorization to access - introduction information (currently symmetric encryption with the - descriptor cookie). - - (2) For performing authorization at the introduction point we can use: - - the fields intro-authorization and service-authorization in - hidden service descriptors, - - a maximum of 215 octets in the ESTABLISH_INTRO cell, and - - one part of 110 octets in the INTRODUCE1V cell. - - (3) For performing authorization at the hidden service we can use: - - the fields intro-authorization and service-authorization in - hidden service descriptors, - - the other part of 110 octets in the INTRODUCE2 cell. - - It will also still be possible to access a hidden service without any - authorization or only use a part of the authorization infrastructure. - However, this requires to consider all parts of the infrastructure. For - example, authorization at the introduction point relying on confidential - intro-authorization data transported in the hidden service descriptor - cannot be performed without using an encryption schema for introduction - information. - - 1.5. Managing authorization data at servers and clients - - In order to provide authorization data at the hidden service and the - authenticated clients, we propose to use files---either the Tor - configuration file or separate files. The exact format of these special - files depends on the authorization protocol used. - - Currently, rend-spec contains the proposition to encode client-side - authorization data in the URL, like in x.y.z.onion. This was never used - and is also a bad idea, because in case of HTTP the requested URL may be - contained in the Host and Referer fields. - - 1.6. Limitations for authorization protocols - - There are two limitations of the current hidden service protocol for - authorization protocols that shall be identified here. - - 1. The three cell types ESTABLISH_INTRO, INTRODUCE1V, and INTRODUCE2 - restricts the amount of data that can be used for authorization. - This forces authorization protocols that require per-user - authorization data at the introduction point to restrict the number - of authorized clients artificially. A possible solution could be to - split contents among multiple cells and reassemble them at the - introduction points. - - 2. The current hidden service protocol does not specify cell types to - perform interactive authorization between client and introduction - point or hidden service. If there should be an authorization - protocol that requires interaction, new cell types would have to be - defined and integrated into the hidden service protocol. - - - 2. Specific authorization protocol instances - - In the following we present two specific authorization protocols that - make use of (parts of) the new authorization infrastructure: - - 1. The first protocol allows a service provider to restrict access - to clients with a previously received secret key only, but does not - attempt to hide service activity from others. - - 2. The second protocol, albeit being feasible for a limited set of about - 16 clients, performs client authorization and hides service activity - from everyone but the authorized clients. - - These two protocol instances extend the existing hidden service protocol - version 2. Hidden services that perform client authorization may run in - parallel to other services running versions 0, 2, or both. - - 2.1. Service with large-scale client authorization - - The first client authorization protocol aims at performing access control - while consuming as few additional resources as possible. A service - provider should be able to permit access to a large number of clients - while denying access for everyone else. However, the price for - scalability is that the service won't be able to hide its activity from - unauthorized or formerly authorized clients. - - The main idea of this protocol is to encrypt the introduction-point part - in hidden service descriptors to authorized clients using symmetric keys. - This ensures that nobody else but authorized clients can learn which - introduction points a service currently uses, nor can someone send a - valid INTRODUCE1 message without knowing the introduction key. Therefore, - a subsequent authorization at the introduction point is not required. - - A service provider generates symmetric "descriptor cookies" for his - clients and distributes them outside of Tor. The suggested key size is - 128 bits, so that descriptor cookies can be encoded in 22 base64 chars - (which can hold up to 22 * 5 = 132 bits, leaving 4 bits to encode the - authorization type (here: "0") and allow a client to distinguish this - authorization protocol from others like the one proposed below). - Typically, the contact information for a hidden service using this - authorization protocol looks like this: - - v2cbb2l4lsnpio4q.onion Ll3X7Xgz9eHGKCCnlFH0uz - - When generating a hidden service descriptor, the service encrypts the - introduction-point part with a single randomly generated symmetric - 128-bit session key using AES-CTR as described for v2 hidden service - descriptors in rend-spec. Afterwards, the service encrypts the session - key to all descriptor cookies using AES. Authorized client should be able - to efficiently find the session key that is encrypted for him/her, so - that 4 octet long client ID are generated consisting of descriptor cookie - and initialization vector. Descriptors always contain a number of - encrypted session keys that is a multiple of 16 by adding fake entries. - Encrypted session keys are ordered by client IDs in order to conceal - addition or removal of authorized clients by the service provider. - - ATYPE Authorization type: set to 1. [1 octet] - ALEN Number of clients := 1 + ((clients - 1) div 16) [1 octet] - for each symmetric descriptor cookie: - ID Client ID: H(descriptor cookie | IV)[:4] [4 octets] - SKEY Session key encrypted with descriptor cookie [16 octets] - (end of client-specific part) - RND Random data [(15 - ((clients - 1) mod 16)) * 20 octets] - IV AES initialization vector [16 octets] - IPOS Intro points, encrypted with session key [remaining octets] - - An authorized client needs to configure Tor to use the descriptor cookie - when accessing the hidden service. Therefore, a user adds the contact - information that she received from the service provider to her torrc - file. Upon downloading a hidden service descriptor, Tor finds the - encrypted introduction-point part and attempts to decrypt it using the - configured descriptor cookie. (In the rare event of two or more client - IDs being equal a client tries to decrypt all of them.) - - Upon sending the introduction, the client includes her descriptor cookie - as auth type "1" in the INTRODUCE2 cell that she sends to the service. - The hidden service checks whether the included descriptor cookie is - authorized to access the service and either responds to the introduction - request, or not. - - 2.2. Authorization for limited number of clients - - A second, more sophisticated client authorization protocol goes the extra - mile of hiding service activity from unauthorized clients. With all else - being equal to the preceding authorization protocol, the second protocol - publishes hidden service descriptors for each user separately and gets - along with encrypting the introduction-point part of descriptors to a - single client. This allows the service to stop publishing descriptors for - removed clients. As long as a removed client cannot link descriptors - issued for other clients to the service, it cannot derive service - activity any more. The downside of this approach is limited scalability. - Even though the distributed storage of descriptors (cf. proposal 114) - tackles the problem of limited scalability to a certain extent, this - protocol should not be used for services with more than 16 clients. (In - fact, Tor should refuse to advertise services for more than this number - of clients.) - - A hidden service generates an asymmetric "client key" and a symmetric - "descriptor cookie" for each client. The client key is used as - replacement for the service's permanent key, so that the service uses a - different identity for each of his clients. The descriptor cookie is used - to store descriptors at changing directory nodes that are unpredictable - for anyone but service and client, to encrypt the introduction-point - part, and to be included in INTRODUCE2 cells. Once the service has - created client key and descriptor cookie, he tells them to the client - outside of Tor. The contact information string looks similar to the one - used by the preceding authorization protocol (with the only difference - that it has "1" encoded as auth-type in the remaining 4 of 132 bits - instead of "0" as before). - - When creating a hidden service descriptor for an authorized client, the - hidden service uses the client key and descriptor cookie to compute - secret ID part and descriptor ID: - - secret-id-part = H(time-period | descriptor-cookie | replica) - - descriptor-id = H(client-key[:10] | secret-id-part) - - The hidden service also replaces permanent-key in the descriptor with - client-key and encrypts introduction-points with the descriptor cookie. - - ATYPE Authorization type: set to 2. [1 octet] - IV AES initialization vector [16 octets] - IPOS Intro points, encr. with descriptor cookie [remaining octets] - - When uploading descriptors, the hidden service needs to make sure that - descriptors for different clients are not uploaded at the same time (cf. - Section 1.1) which is also a limiting factor for the number of clients. - - When a client is requested to establish a connection to a hidden service - it looks up whether it has any authorization data configured for that - service. If the user has configured authorization data for authorization - protocol "2", the descriptor ID is determined as described in the last - paragraph. Upon receiving a descriptor, the client decrypts the - introduction-point part using its descriptor cookie. Further, the client - includes its descriptor cookie as auth-type "2" in INTRODUCE2 cells that - it sends to the service. - - 2.3. Hidden service configuration - - A hidden service that is meant to perform client authorization adds a - new option HiddenServiceAuthorizeClient to its hidden service - configuration. This option contains the authorization type which is - either "1" for the protocol described in 2.1 or "2" for the protocol in - 2.2 and a comma-separated list of human-readable client names, so that - Tor can create authorization data for these clients: - - HiddenServiceAuthorizeClient auth-type client-name,client-name,... - - If this option is configured, HiddenServiceVersion is automatically - reconfigured to contain only version numbers of 2 or higher. - - Tor stores all generated authorization data for the authorization - protocols described in Sections 2.1 and 2.2 in a new file using the - following file format: - - "client-name" human-readable client identifier NL - "descriptor-cookie" 128-bit key ^= 22 base64 chars NL - - If the authorization protocol of Section 2.2 is used, Tor also generates - and stores the following data: - - "client-key" NL a public key in PEM format - - 2.4. Client configuration - - Clients need to make their authorization data known to Tor using another - configuration option that contains a service name (mainly for the sake of - convenience), the service address, and the descriptor cookie that is - required to access a hidden service (the authorization protocol number is - encoded in the descriptor cookie): - - HidServAuth service-name service-address descriptor-cookie - -Security implications: - - In the following we want to discuss possible attacks by dishonest - entities in the presented infrastructure and specific protocol. These - security implications would have to be verified once more when adding - another protocol. The dishonest entities (theoretically) include the - hidden service itself, the authenticated clients, hidden service directory - nodes, introduction points, and rendezvous points. The relays that are - part of circuits used during protocol execution, but never learn about - the exchanged descriptors or cells by design, are not considered. - Obviously, this list makes no claim to be complete. The discussed attacks - are sorted by the difficulty to perform them, in ascending order, - starting with roles that everyone could attempt to take and ending with - partially trusted entities abusing the trust put in them. - - (1) A hidden service directory could attempt to conclude presence of a - service from the existence of a locally stored hidden service descriptor: - This passive attack is possible only for a single client-service - relation, because descriptors need to contain a publicly visible - signature of the service using the client key. - A possible protection would be to increase the number of hidden service - directories in the network. - - (2) A hidden service directory could try to break the descriptor cookies - of locally stored descriptors: This attack can be performed offline. The - only useful countermeasure against it might be using safe passwords that - are generated by Tor. - -[passwords? where did those come in? -RD] - - (3) An introduction point could try to identify the pseudonym of the - hidden service on behalf of which it operates: This is impossible by - design, because the service uses a fresh public key for every - establishment of an introduction point (see proposal 114) and the - introduction point receives a fresh introduction cookie, so that there is - no identifiable information about the service that the introduction point - could learn. The introduction point cannot even tell if client accesses - belong to the same client or not, nor can it know the total number of - authorized clients. The only information might be the pattern of - anonymous client accesses, but that is hardly enough to reliably identify - a specific service. - - (4) An introduction point could want to learn the identities of accessing - clients: This is also impossible by design, because all clients use the - same introduction cookie for authorization at the introduction point. - - (5) An introduction point could try to replay a correct INTRODUCE1 cell - to other introduction points of the same service, e.g. in order to force - the service to create a huge number of useless circuits: This attack is - not possible by design, because INTRODUCE1 cells are encrypted using a - freshly created introduction key that is only known to authorized - clients. - - (6) An introduction point could attempt to replay a correct INTRODUCE2 - cell to the hidden service, e.g. for the same reason as in the last - attack: This attack is stopped by the fact that a service will drop - INTRODUCE2 cells containing a DH handshake they have seen recently. - - (7) An introduction point could block client requests by sending either - positive or negative INTRODUCE_ACK cells back to the client, but without - forwarding INTRODUCE2 cells to the server: This attack is an annoyance - for clients, because they might wait for a timeout to elapse until trying - another introduction point. However, this attack is not introduced by - performing authorization and it cannot be targeted towards a specific - client. A countermeasure might be for the server to periodically perform - introduction requests to his own service to see if introduction points - are working correctly. - - (8) The rendezvous point could attempt to identify either server or - client: This remains impossible as it was before, because the - rendezvous cookie does not contain any identifiable information. - - (9) An authenticated client could swamp the server with valid INTRODUCE1 - and INTRODUCE2 cells, e.g. in order to force the service to create - useless circuits to rendezvous points; as opposed to an introduction - point replaying the same INTRODUCE2 cell, a client could include a new - rendezvous cookie for every request: The countermeasure for this attack - is the restriction to 10 connection establishments per client per hour. - -Compatibility: - - An implementation of this proposal would require changes to hidden - services and clients to process authorization data and encode and - understand the new formats. However, both services and clients would - remain compatible to regular hidden services without authorization. - -Implementation: - - The implementation of this proposal can be divided into a number of - changes to hidden service and client side. There are no - changes necessary on directory, introduction, or rendezvous nodes. All - changes are marked with either [service] or [client] do denote on which - side they need to be made. - - /1/ Configure client authorization [service] - - - Parse configuration option HiddenServiceAuthorizeClient containing - authorized client names. - - Load previously created client keys and descriptor cookies. - - Generate missing client keys and descriptor cookies, add them to - client_keys file. - - Rewrite the hostname file. - - Keep client keys and descriptor cookies of authorized clients in - memory. - [- In case of reconfiguration, mark which client authorizations were - added and whether any were removed. This can be used later when - deciding whether to rebuild introduction points and publish new - hidden service descriptors. Not implemented yet.] - - /2/ Publish hidden service descriptors [service] - - - Create and upload hidden service descriptors for all authorized - clients. - [- See /1/ for the case of reconfiguration.] - - /3/ Configure permission for hidden services [client] - - - Parse configuration option HidServAuth containing service - authorization, store authorization data in memory. - - /5/ Fetch hidden service descriptors [client] - - - Look up client authorization upon receiving a hidden service request. - - Request hidden service descriptor ID including client key and - descriptor cookie. Only request v2 descriptors, no v0. - - /6/ Process hidden service descriptor [client] - - - Decrypt introduction points with descriptor cookie. - - /7/ Create introduction request [client] - - - Include descriptor cookie in INTRODUCE2 cell to introduction point. - - Pass descriptor cookie around between involved connections and - circuits. - - /8/ Process introduction request [service] - - - Read descriptor cookie from INTRODUCE2 cell. - - Check whether descriptor cookie is authorized for access, including - checking access counters. - - Log access for accountability. - diff --git a/doc/spec/proposals/122-unnamed-flag.txt b/doc/spec/proposals/122-unnamed-flag.txt deleted file mode 100644 index 6502b9c56..000000000 --- a/doc/spec/proposals/122-unnamed-flag.txt +++ /dev/null @@ -1,138 +0,0 @@ -Filename: 122-unnamed-flag.txt -Title: Network status entries need a new Unnamed flag -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 04-Oct-2007 -Status: Closed -Implemented-In: 0.2.0.x - -1. Overview: - - Tor's directory authorities can give certain servers a "Named" flag - in the network-status entry, when they want to bind that nickname to - that identity key. This allows clients to specify a nickname rather - than an identity fingerprint and still be certain they're getting the - "right" server. As dir-spec.txt describes it, - - Name X is bound to identity Y if at least one binding directory lists - it, and no directory binds X to some other Y'. - - In practice, clients can refer to servers by nickname whether they are - Named or not; if they refer to nicknames that aren't Named, a complaint - shows up in the log asking them to use the identity key in the future - --- but it still works. - - The problem? Imagine a Tor server with nickname Bob. Bob and his - identity fingerprint are registered in tor26's approved-routers - file, but none of the other authorities registered him. Imagine - there are several other unregistered servers also with nickname Bob - ("the imposters"). - - While Bob is online, all is well: a) tor26 gives a Named flag to - the real one, and refuses to list the other ones; and b) the other - authorities list the imposters but don't give them a Named flag. Clients - who have all the network-statuses can compute which one is the real Bob. - - But when the real Bob disappears and his descriptor expires? tor26 - continues to refuse to list any of the imposters, and the other - authorities continue to list the imposters. Clients don't have any - idea that there exists a Named Bob, so they can ask for server Bob and - get one of the imposters. (A warning will also appear in their log, - but so what.) - -2. The stopgap solution: - - tor26 should start accepting and listing the imposters, but it should - assign them a new flag: "Unnamed". - - This would produce three cases in terms of assigning flags in the consensus - networkstatus: - - i) a router gets the Named flag in the v3 networkstatus if - a) it's the only router with that nickname that has the Named flag - out of all the votes, and - b) no vote lists it as Unnamed - else, - ii) a router gets the Unnamed flag if - a) some vote lists a different router with that nickname as Named, or - b) at least one vote lists it as Unnamed, or - c) there are other routers with the same nickname that are Unnamed - else, - iii) the router neither gets a Named nor an Unnamed flag. - - (This whole proposal is meant only for v3 dir flags; we shouldn't try - to backport it to the v2 dir world.) - - Then client behavior is: - - a) If there's a Bob with a Named flag, pick that one. - else b) If the Bobs don't have the Unnamed flag (notice that they should - either all have it, or none), pick one of them and warn. - else c) They all have the Unnamed flag -- no router found. - -3. Problems not solved by this stopgap: - - 3.1. Naming authorities can go offline. - - If tor26 is the only authority that provides a binding for Bob, when - tor26 goes offline we're back in our previous situation -- the imposters - can be referenced with a mere ignorable warning in the client's log. - - If some other authority Names a different Bob, and tor26 goes offline, - then that other Bob becomes the unique Named Bob. - - So be it. We should try to solve these one day, but there's no clear way - to do it that doesn't destroy usability in other ways, and if we want - to get the Unnamed flag into v3 network statuses we should add it soon. - - 3.2. V3 dir spec magnifies brief discrepancies. - - Another point to notice is if tor26 names Bob(1), doesn't know about - Bob(2), but moria lists Bob(2). Then Bob(2) doesn't get an Unnamed flag - even if it should (and Bob(1) is not around). - - Right now, in v2 dirs, the case where an authority doesn't know about - a server but the other authorities do know is rare. That's because - authorities periodically ask for other networkstatuses and then fetch - descriptors that are missing. - - With v3, if that window occurs at the wrong time, it is extended for the - entire period. We could solve this by making the voting more complex, - but that doesn't seem worth it. - - [3.3. Tor26 is only one tor26. - - We need more naming authorities, possibly with some kind of auto-naming - feature. This is out-of-scope for this proposal -NM] - -4. Changes to the v2 directory - - Previously, v2 authorities that had a binding for a server named Bob did - not list any other server named Bob. This will change too: - - Version 2 authorities will start listing all routers they know about, - whether they conflict with a name-binding or not: Servers for which - this authority has a binding will continue to be marked Named, - additionally all other servers of that nickname will be listed without the - Named flag (i.e. there will be no Unnamed flag in v2 status documents). - - Clients already should handle having a named Bob alongside unnamed - Bobs correctly, and having the unnamed Bobs in the status file even - without the named server is no worse than the current status quo where - clients learn about those servers from other authorities. - - The benefit of this is that an authority's opinion on a server like - Guard, Stable, Fast etc. can now be learned by clients even if that - specific authority has reserved that server's name for somebody else. - -5. Other benefits: - - This new flag will allow people to operate servers that happen to have - the same nickname as somebody who registered their server two years ago - and left soon after. Right now there are dozens of nicknames that are - registered on all three binding directory authorities, yet haven't been - running for years. While it's bad that these nicknames are effectively - blacklisted from the network, the really bad part is that this logic - is really unintuitive to prospective new server operators. - diff --git a/doc/spec/proposals/123-autonaming.txt b/doc/spec/proposals/123-autonaming.txt deleted file mode 100644 index 6cd25329f..000000000 --- a/doc/spec/proposals/123-autonaming.txt +++ /dev/null @@ -1,56 +0,0 @@ -Filename: 123-autonaming.txt -Title: Naming authorities automatically create bindings -Version: $Revision$ -Last-Modified: $Date$ -Author: Peter Palfrader -Created: 2007-10-11 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - Tor's directory authorities can give certain servers a "Named" flag - in the network-status entry, when they want to bind that nickname to - that identity key. This allows clients to specify a nickname rather - than an identity fingerprint and still be certain they're getting the - "right" server. - - Authority operators name a server by adding their nickname and - identity fingerprint to the 'approved-routers' file. Historically - being listed in the file was required for a router, at first for being - listed in the directory at all, and later in order to be used by - clients as a first or last hop of a circuit. - - Adding identities to the list of named routers so far has been a - manual, time consuming, and boring job. Given that and the fact that - the Tor network works just fine without named routers the last - authority to keep a current binding list stopped updating it well over - half a year ago. - - Naming, if it were done, would serve a useful purpose however in that - users can have a reasonable expectation that the exit server Bob they - are using in their http://www.google.com.bob.exit/ URL is the same - Bob every time. - -Proposal: - I propose that identity<->name binding be completely automated: - - New bindings should be added after the router has been around for a - bit and their name has not been used by other routers, similarly names - that have not appeared on the network for a long time should be freed - in case a new router wants to use it. - - The following rules are suggested: - i) If a named router has not been online for half a year, the - identity<->name binding for that name is removed. The nickname - is free to be taken by other routers now. - ii) If a router claims a certain nickname and - a) has been on the network for at least two weeks, and - b) that nickname is not yet linked to a different router, and - c) no other router has wanted that nickname in the last month, - a new binding should be created for this router and its desired - nickname. - - This automaton does not necessarily need to live in the Tor code, it - can do its job just as well when it's an external tool. - diff --git a/doc/spec/proposals/124-tls-certificates.txt b/doc/spec/proposals/124-tls-certificates.txt deleted file mode 100644 index 0a4777273..000000000 --- a/doc/spec/proposals/124-tls-certificates.txt +++ /dev/null @@ -1,315 +0,0 @@ -Filename: 124-tls-certificates.txt -Title: Blocking resistant TLS certificate usage -Version: $Revision$ -Last-Modified: $Date$ -Author: Steven J. Murdoch -Created: 2007-10-25 -Status: Superseded - -Overview: - - To be less distinguishable from HTTPS web browsing, only Tor servers should - present TLS certificates. This should be done whilst maintaining backwards - compatibility with Tor nodes which present and expect client certificates, and - while preserving existing security properties. This specification describes - the negotiation protocol, what certificates should be presented during the TLS - negotiation, and how to move the client authentication within the encrypted - tunnel. - -Motivation: - - In Tor's current TLS [1] handshake, both client and server present a - two-certificate chain. Since TLS performs authentication prior to establishing - the encrypted tunnel, the contents of these certificates are visible to an - eavesdropper. In contrast, during normal HTTPS web browsing, the server - presents a single certificate, signed by a root CA and the client presents no - certificate. Hence it is possible to distinguish Tor from HTTP by identifying - this pattern. - - To resist blocking based on traffic identification, Tor should behave as close - to HTTPS as possible, i.e. servers should offer a single certificate and not - request a client certificate; clients should present no certificate. This - presents two difficulties: clients are no longer authenticated and servers are - authenticated by the connection key, rather than identity key. The link - protocol must thus be modified to preserve the old security semantics. - - Finally, in order to maintain backwards compatibility, servers must correctly - identify whether the client supports the modified certificate handling. This - is achieved by modifying the cipher suites that clients advertise support - for. These cipher suites are selected to be similar to those chosen by web - browsers, in order to resist blocking based on client hello. - -Terminology: - - Initiator: OP or OR which initiates a TLS connection ("client" in TLS - terminology) - - Responder: OR which receives an incoming TLS connection ("server" in TLS - terminology) - -Version negotiation and cipher suite selection: - - In the modified TLS handshake, the responder does not request a certificate - from the initiator. This request would normally occur immediately after the - responder receives the client hello (the first message in a TLS handshake) and - so the responder must decide whether to request a certificate based only on - the information in the client hello. This is achieved by examining the cipher - suites in the client hello. - - List 1: cipher suites lists offered by version 0/1 Tor - - From src/common/tortls.c, revision 12086: - TLS1_TXT_DHE_RSA_WITH_AES_128_SHA - TLS1_TXT_DHE_RSA_WITH_AES_128_SHA : SSL3_TXT_EDH_RSA_DES_192_CBC3_SHA - SSL3_TXT_EDH_RSA_DES_192_CBC3_SHA - - Client hello sent by initiator: - - Initiators supporting version 2 of the Tor connection protocol MUST - offer a different cipher suite list from those sent by pre-version 2 - Tors, contained in List 1. To maintain compatibility with older Tor - versions and common browsers, the cipher suite list MUST include - support for: - - TLS_DHE_RSA_WITH_AES_256_CBC_SHA - TLS_DHE_RSA_WITH_AES_128_CBC_SHA - SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA - SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA - - Client hello received by responder/server hello sent by responder: - - Responders supporting version 2 of the Tor connection protocol should compare - the cipher suite list in the client hello with those in List 1. If it matches - any in the list then the responder should assume that the initiatior supports - version 1, and thus should maintain the version 1 behavior, i.e. send a - two-certificate chain, request a client certificate and do not send or expect - a VERSIONS cell [2]. - - Otherwise, the responder should assume version 2 behavior and select a cipher - suite following TLS [1] behavior, i.e. select the first entry from the client - hello cipher list which is acceptable. Responders MUST NOT select any suite - that lacks ephemeral keys, or whose symmetric keys are less then KEY_LEN bits, - or whose digests are less than HASH_LEN bits. Implementations SHOULD NOT - allow other SSLv3 ciphersuites. - - Should no mutually acceptable cipher suite be found, the connection MUST be - closed. - - If the responder is implementing version 2 of the connection protocol it - SHOULD send a server certificate with random contents. The organizationName - field MUST NOT be "Tor", "TOR" or "t o r". - - Server certificate received by initiator: - - If the server certificate has an organizationName of "Tor", "TOR" or "t o r", - the initiator should assume that the responder does not support version 2 of - the connection protocol. In which case the initiator should respond following - version 1, i.e. send a two-certificate client chain and do not send or expect - a VERSIONS cell. - - [SJM: We could also use the fact that a client certificate request was sent] - - If the server hello contains a ciphersuite which does not comply with the key - length requirements above, even if it was one offered in the client hello, the - connection MUST be closed. This will only occur if the responder is not a Tor - server. - - Backward compatibility: - - v1 Initiator, v1 Responder: No change - v1 Initiator, v2 Responder: Responder detects v1 initiator by client hello - v2 Initiator, v1 Responder: Responder accepts v2 client hello. Initiator - detects v1 server certificate and continues with v1 protocol - v2 Initiator, v2 Responder: Responder accepts v2 client hello. Initiator - detects v2 server certificate and continues with v2 protocol. - - Additional link authentication process: - - Following VERSION and NETINFO negotiation, both responder and - initiator MUST send a certification chain in a CERT cell. If one - party does not have a certificate, the CERT cell MUST still be sent, - but with a length of zero. - - A CERT cell is a variable length cell, of the format - CircID [2 bytes] - Command [1 byte] - Length [2 bytes] - Payload [ bytes] - - CircID MUST set to be 0x0000 - Command is [SJM: TODO] - Length is the length of the payload - Payload contains 0 or more certificates, each is of the format: - Cert_Length [2 bytes] - Certificate [ bytes] - - Each certificate MUST sign the one preceding it. The initator MUST - place its connection certificate first; the responder, having - already sent its connection certificate as part of the TLS handshake - MUST place its identity certificate first. - - Initiators who send a CERT cell MUST follow that with an LINK_AUTH - cell to prove that they posess the corresponding private key. - - A LINK_AUTH cell is fixed-lenth, of the format: - CircID [2 bytes] - Command [1 byte] - Length [2 bytes] - Payload (padded with 0 bytes) [PAYLOAD_LEN - 2 bytes] - - CircID MUST set to be 0x0000 - Command is [SJM: TODO] - Length is the valid portion of the payload - Payload is of the format: - Signature version [1 byte] - Signature [ - 1 bytes] - Padding [PAYLOAD_LEN - - 2 bytes] - - Signature version: Identifies the type of signature, currently 0x00 - Signature: Digital signature under the initiator's connection key of the - following item, in PKCS #1 block type 1 [3] format: - - HMAC-SHA1, using the TLS master secret as key, of the - following elements concatenated: - - The signature version (0x00) - - The NUL terminated ASCII string: "Tor initiator certificate verification" - - client_random, as sent in the Client Hello - - server_random, as sent in the Server Hello - - SHA-1 hash of the initiator connection certificate - - SHA-1 hash of the responder connection certificate - - Security checks: - - - Before sending a LINK_AUTH cell, a node MUST ensure that the TLS - connection is authenticated by the responder key. - - For the handshake to have succeeded, the initiator MUST confirm: - - That the TLS handshake was authenticated by the - responder connection key - - That the responder connection key was signed by the first - certificate in the CERT cell - - That each certificate in the CERT cell was signed by the - following certificate, with the exception of the last - - That the last certificate in the CERT cell is the expected - identity certificate for the node being connected to - - For the handshake to have succeeded, the responder MUST confirm - either: - A) - A zero length CERT cell was sent and no LINK_AUTH cell was - sent - In which case the responder shall treat the identity of the - initiator as unknown - or - B) - That the LINK_AUTH MAC contains a signature by the first - certificate in the CERT cell - - That the MAC signed matches the expected value - - That each certificate in the CERT cell was signed by the - following certificate, with the exception of the last - In which case the responder shall treat the identity of the - initiator as that of the last certificate in the CERT cell - - Protocol summary: - - 1. I(nitiator) <-> R(esponder): TLS handshake, including responder - authentication under connection certificate R_c - 2. I <->: VERSION and NETINFO negotiation - 3. R -> I: CERT (Responder identity certificate R_i (which signs R_c)) - 4. I -> R: CERT (Initiator connection certificate I_c, - Initiator identity certificate I_i (which signs I_c) - 5. I -> R: LINK_AUTH (Signature, under I_c of HMAC-SHA1(master_secret, - "Tor initiator certificate verification" || - client_random || server_random || - I_c hash || R_c hash) - - Notes: I -> R doesn't need to wait for R_i before sending its own - messages (reduces round-trips). - Certificate hash is calculated like identity hash in CREATE cells. - Initiator signature is calculated in a similar way to Certificate - Verify messages in TLS 1.1 (RFC4346, Sections 7.4.8 and 4.7). - If I is an OP, a zero length certificate chain may be sent in step 4; - In which case, step 5 is not performed - - Rationale: - - - Version and netinfo negotiation before authentication: The version cell needs - to come before before the rest of the protocol, since we may choose to alter - the rest at some later point, e.g switch to a different MAC/signature scheme. - It is useful to keep the NETINFO and VERSION cells close to each other, since - the time between them is used to check if there is a delay-attack. Still, a - server might want to not act on NETINFO data from an initiator until the - authentication is complete. - -Appendix A: Cipher suite choices - - This specification intentionally does not put any constraints on the - TLS ciphersuite lists presented by clients, other than a minimum - required for compatibility. However, to maximize blocking - resistance, ciphersuite lists should be carefully selected. - - Recommended client ciphersuite list - - Source: http://lxr.mozilla.org/security/source/security/nss/lib/ssl/sslproto.h - - 0xc00a: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA - 0xc014: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA - 0x0039: TLS_DHE_RSA_WITH_AES_256_CBC_SHA - 0x0038: TLS_DHE_DSS_WITH_AES_256_CBC_SHA - 0xc00f: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA - 0xc005: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA - 0x0035: TLS_RSA_WITH_AES_256_CBC_SHA - 0xc007: TLS_ECDHE_ECDSA_WITH_RC4_128_SHA - 0xc009: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA - 0xc011: TLS_ECDHE_RSA_WITH_RC4_128_SHA - 0xc013: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA - 0x0033: TLS_DHE_RSA_WITH_AES_128_CBC_SHA - 0x0032: TLS_DHE_DSS_WITH_AES_128_CBC_SHA - 0xc00c: TLS_ECDH_RSA_WITH_RC4_128_SHA - 0xc00e: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA - 0xc002: TLS_ECDH_ECDSA_WITH_RC4_128_SHA - 0xc004: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA - 0x0004: SSL_RSA_WITH_RC4_128_MD5 - 0x0005: SSL_RSA_WITH_RC4_128_SHA - 0x002f: TLS_RSA_WITH_AES_128_CBC_SHA - 0xc008: TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA - 0xc012: TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA - 0x0016: SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA - 0x0013: SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA - 0xc00d: TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA - 0xc003: TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA - 0xfeff: SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA (168-bit Triple DES with RSA and a SHA1 MAC) - 0x000a: SSL_RSA_WITH_3DES_EDE_CBC_SHA - - Order specified in: - http://lxr.mozilla.org/security/source/security/nss/lib/ssl/sslenum.c#47 - - Recommended options: - 0x0000: Server Name Indication [4] - 0x000a: Supported Elliptic Curves [5] - 0x000b: Supported Point Formats [5] - - Recommended compression: - 0x00 - - Recommended server ciphersuite selection: - - The responder should select the first entry in this list which is - listed in the client hello: - - 0x0039: TLS_DHE_RSA_WITH_AES_256_CBC_SHA [ Common Firefox choice ] - 0x0033: TLS_DHE_RSA_WITH_AES_128_CBC_SHA [ Tor v1 default ] - 0x0016: SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA [ Tor v1 fallback ] - 0x0013: SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA [ Valid IE option ] - -References: - -[1] The Transport Layer Security (TLS) Protocol, Version 1.1, RFC4346, IETF - -[2] Version negotiation for the Tor protocol, Tor proposal 105 - -[3] B. Kaliski, "Public-Key Cryptography Standards (PKCS) #1: - RSA Cryptography Specifications Version 1.5", RFC 2313, - March 1998. - -[4] TLS Extensions, RFC 3546 - -[5] Elliptic Curve Cryptography (ECC) Cipher Suites for Transport Layer Security (TLS) - -% diff --git a/doc/spec/proposals/125-bridges.txt b/doc/spec/proposals/125-bridges.txt deleted file mode 100644 index 8bb316978..000000000 --- a/doc/spec/proposals/125-bridges.txt +++ /dev/null @@ -1,293 +0,0 @@ -Filename: 125-bridges.txt -Title: Behavior for bridge users, bridge relays, and bridge authorities -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 11-Nov-2007 -Status: Closed -Implemented-In: 0.2.0.x - -0. Preface - - This document describes the design decisions around support for bridge - users, bridge relays, and bridge authorities. It acts as an overview - of the bridge design and deployment for developers, and it also tries - to point out limitations in the current design and implementation. - - For more details on what all of these mean, look at blocking.tex in - /doc/design-paper/ - -1. Bridge relays - - Bridge relays are just like normal Tor relays except they don't publish - their server descriptors to the main directory authorities. - -1.1. PublishServerDescriptor - - To configure your relay to be a bridge relay, just add - BridgeRelay 1 - PublishServerDescriptor bridge - to your torrc. This will cause your relay to publish its descriptor - to the bridge authorities rather than to the default authorities. - - Alternatively, you can say - BridgeRelay 1 - PublishServerDescriptor 0 - which will cause your relay to not publish anywhere. This could be - useful for private bridges. - -1.2. Exit policy - - Bridge relays should use an exit policy of "reject *:*". This is - because they only need to relay traffic between the bridge users - and the rest of the Tor network, so there's no need to let people - exit directly from them. - -1.3. RelayBandwidthRate / RelayBandwidthBurst - - We invented the RelayBandwidth* options for this situation: Tor clients - who want to allow relaying too. See proposal 111 for details. Relay - operators should feel free to rate-limit their relayed traffic. - -1.4. Helping the user with port forwarding, NAT, etc. - - Just as for operating normal relays, our documentation and hints for - how to make your ORPort reachable are inadequate for normal users. - - We need to work harder on this step, perhaps in 0.2.2.x. - -1.5. Vidalia integration - - Vidalia has turned its "Relay" settings page into a tri-state - "Don't relay" / "Relay for the Tor network" / "Help censored users". - - If you click the third choice, it forces your exit policy to reject *:*. - - If all the bridges end up on port 9001, that's not so good. On the - other hand, putting the bridges on a low-numbered port in the Unix - world requires jumping through extra hoops. The current compromise is - that Vidalia makes the ORPort default to 443 on Windows, and 9001 on - other platforms. - - At the bottom of the relay config settings window, Vidalia displays - the bridge identifier to the operator (see Section 3.1) so he can pass - it on to bridge users. - -1.6. What if the default ORPort is already used? - - If the user already has a webserver or some other application - bound to port 443, then Tor will fail to bind it and complain to the - user, probably in a cryptic way. Rather than just working on a better - error message (though we should do this), we should consider an - "ORPort auto" option that tells Tor to try to find something that's - bindable and reachable. This would also help us tolerate ISPs that - filter incoming connections on port 80 and port 443. But this should - be a different proposal, and can wait until 0.2.2.x. - -2. Bridge authorities. - - Bridge authorities are like normal directory authorities, except they - don't create their own network-status documents or votes. So if you - ask an authority for a network-status document or consensus, they - behave like a directory mirror: they give you one from one of the main - authorities. But if you ask the bridge authority for the descriptor - corresponding to a particular identity fingerprint, it will happily - give you the latest descriptor for that fingerprint. - - To become a bridge authority, add these lines to your torrc: - AuthoritativeDirectory 1 - BridgeAuthoritativeDir 1 - - Right now there's one bridge authority, running on the Tonga relay. - -2.1. Exporting bridge-purpose descriptors - - We've added a new purpose for server descriptors: the "bridge" - purpose. With the new router-descriptors file format that includes - annotations, it's easy to look through it and find the bridge-purpose - descriptors. - - Currently we export the bridge descriptors from Tonga to the - BridgeDB server, so it can give them out according to the policies - in blocking.pdf. - -2.2. Reachability/uptime testing - - Right now the bridge authorities do active reachability testing of - bridges, so we know which ones to recommend for users. - - But in the design document, we suggested that bridges should publish - anonymously (i.e. via Tor) to the bridge authority, so somebody watching - the bridge authority can't just enumerate all the bridges. But if we're - doing active measurement, the game is up. Perhaps we should back off on - this goal, or perhaps we should do our active measurement anonymously? - - Answering this issue is scheduled for 0.2.1.x. - -2.3. Migrating to multiple bridge authorities - - Having only one bridge authority is both a trust bottleneck (if you - break into one place you learn about every single bridge we've got) - and a robustness bottleneck (when it's down, bridge users become sad). - - Right now if we put up a second bridge authority, all the bridges would - publish to it, and (assuming the code works) bridge users would query - a random bridge authority. This resolves the robustness bottleneck, - but makes the trust bottleneck even worse. - - In 0.2.2.x and later we should think about better ways to have multiple - bridge authorities. - -3. Bridge users. - - Bridge users are like ordinary Tor users except they use encrypted - directory connections by default, and they use bridge relays as both - entry guards (their first hop) and directory guards (the source of - all their directory information). - - To become a bridge user, add the following line to your torrc: - - UseBridges 1 - - and then add at least one "Bridge" line to your torrc based on the - format below. - -3.1. Format of the bridge identifier. - - The canonical format for a bridge identifier contains an IP address, - an ORPort, and an identity fingerprint: - bridge 128.31.0.34:9009 4C17 FB53 2E20 B2A8 AC19 9441 ECD2 B017 7B39 E4B1 - - However, the identity fingerprint can be left out, in which case the - bridge user will connect to that relay and use it as a bridge regardless - of what identity key it presents: - bridge 128.31.0.34:9009 - This might be useful for cases where only short bridge identifiers - can be communicated to bridge users. - - In a future version we may also support bridge identifiers that are - only a key fingerprint: - bridge 4C17 FB53 2E20 B2A8 AC19 9441 ECD2 B017 7B39 E4B1 - and the bridge user can fetch the latest descriptor from the bridge - authority (see Section 3.4). - -3.2. Bridges as entry guards - - For now, bridge users add their bridge relays to their list of "entry - guards" (see path-spec.txt for background on entry guards). They are - managed by the entry guard algorithms exactly as if they were a normal - entry guard -- their keys and timing get cached in the "state" file, - etc. This means that when the Tor user starts up with "UseBridges" - disabled, he will skip past the bridge entries since they won't be - listed as up and usable in his networkstatus consensus. But to be clear, - the "entry_guards" list doesn't currently distinguish guards by purpose. - - Internally, each bridge user keeps a smartlist of "bridge_info_t" - that reflects the "bridge" lines from his torrc along with a download - schedule (see Section 3.5 below). When he starts Tor, he attempts - to fetch a descriptor for each configured bridge (see Section 3.4 - below). When he succeeds at getting a descriptor for one of the bridges - in his list, he adds it directly to the entry guard list using the - normal add_an_entry_guard() interface. Once a bridge descriptor has - been added, should_delay_dir_fetches() will stop delaying further - directory fetches, and the user begins to bootstrap his directory - information from that bridge (see Section 3.3). - - Currently bridge users cache their bridge descriptors to the - "cached-descriptors" file (annotated with purpose "bridge"), but - they don't make any attempt to reuse descriptors they find in this - file. The theory is that either the bridge is available now, in which - case you can get a fresh descriptor, or it's not, in which case an - old descriptor won't do you much good. - - We could disable writing out the bridge lines to the state file, if - we think this is a problem. - - As an exception, if we get an application request when we have one - or more bridge descriptors but we believe none of them are running, - we mark them all as running again. This is similar to the exception - already in place to help long-idle Tor clients realize they should - fetch fresh directory information rather than just refuse requests. - -3.3. Bridges as directory guards - - In addition to using bridges as the first hop in their circuits, bridge - users also use them to fetch directory updates. Other than initial - bootstrapping to find a working bridge descriptor (see Section 3.4 - below), all further non-anonymized directory fetches will be redirected - to the bridge. - - This means that bridge relays need to have cached answers for all - questions the bridge user might ask. This makes the upgrade path - tricky --- for example, if we migrate to a v4 directory design, the - bridge user would need to keep using v3 so long as his bridge relays - only knew how to answer v3 queries. - - In a future design, for cases where the user has enough information - to build circuits yet the chosen bridge doesn't know how to answer a - given query, we might teach bridge users to make an anonymized request - to a more suitable directory server. - -3.4. How bridge users get their bridge descriptor - - Bridge users can fetch bridge descriptors in two ways: by going directly - to the bridge and asking for "/tor/server/authority", or by going to - the bridge authority and asking for "/tor/server/fp/ID". By default, - they will only try the direct queries. If the user sets - UpdateBridgesFromAuthority 1 - in his config file, then he will try querying the bridge authority - first for bridges where he knows a digest (if he only knows an IP - address and ORPort, then his only option is a direct query). - - If the user has at least one working bridge, then he will do further - queries to the bridge authority through a full three-hop Tor circuit. - But when bootstrapping, he will make a direct begin_dir-style connection - to the bridge authority. - - As of Tor 0.2.0.10-alpha, if the user attempts to fetch a descriptor - from the bridge authority and it returns a 404 not found, the user - will automatically fall back to trying a direct query. Therefore it is - recommended that bridge users always set UpdateBridgesFromAuthority, - since at worst it will delay their fetches a little bit and notify - the bridge authority of the identity fingerprint (but not location) - of their intended bridges. - -3.5. Bridge descriptor retry schedule - - Bridge users try to fetch a descriptor for each bridge (using the - steps in Section 3.4 above) on startup. Whenever they receive a - bridge descriptor, they reschedule a new descriptor download for 1 - hour from then. - - If on the other hand it fails, they try again after 15 minutes for the - first attempt, after 15 minutes for the second attempt, and after 60 - minutes for subsequent attempts. - - In 0.2.2.x we should come up with some smarter retry schedules. - -3.6. Vidalia integration - - Vidalia 0.0.16 has a checkbox in its Network config window called - "My ISP blocks connections to the Tor network." Users who click that - box change their configuration to: - UseBridges 1 - UpdateBridgesFromAuthority 1 - and should specify at least one Bridge identifier. - -3.7. Do we need a second layer of entry guards? - - If the bridge user uses the bridge as its entry guard, then the - triangulation attacks from Lasse and Paul's Oakland paper work to - locate the user's bridge(s). - - Worse, this is another way to enumerate bridges: if the bridge users - keep rotating through second hops, then if you run a few fast servers - (and avoid getting considered an Exit or a Guard) you'll quickly get - a list of the bridges in active use. - - That's probably the strongest reason why bridge users will need to - pick second-layer guards. Would this mean bridge users should switch - to four-hop circuits? - - We should figure this out in the 0.2.1.x timeframe. - diff --git a/doc/spec/proposals/126-geoip-reporting.txt b/doc/spec/proposals/126-geoip-reporting.txt deleted file mode 100644 index d48a08ba3..000000000 --- a/doc/spec/proposals/126-geoip-reporting.txt +++ /dev/null @@ -1,412 +0,0 @@ -Filename: 126-geoip-reporting.txt -Title: Getting GeoIP data and publishing usage summaries -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 2007-11-24 -Status: Closed -Implemented-In: 0.2.0.x - -0. Status - - In 0.2.0.x, this proposal is implemented to the extent needed to - address its motivations. See notes below with the test "RESOLUTION" - for details. - -1. Background and motivation - - Right now we can keep a rough count of Tor users, both total and by - country, by watching connections to a single directory mirror. Being - able to get usage estimates is useful both for our funders (to - demonstrate progress) and for our own development (so we know how - quickly we're scaling and can design accordingly, and so we know which - countries and communities to focus on more). This need for information - is the only reason we haven't deployed "directory guards" (think of - them like entry guards but for directory information; in practice, - it would seem that Tor clients should simply use their entry guards - as their directory guards; see also proposal 125). - - With the move toward bridges, we will no longer be able to track Tor - clients that use bridges, since they use their bridges as directory - guards. Further, we need to be able to learn which bridges stop seeing - use from certain countries (and are thus likely blocked), so we can - avoid giving them out to other users in those countries. - - Right now we already do GeoIP lookups in Vidalia: Vidalia draws relays - and circuits on its 'network map', and it performs anonymized GeoIP - lookups to its central servers to know where to put the dots. Vidalia - caches answers it gets -- to reduce delay, to reduce overhead on - the network, and to reduce anonymity issues where users reveal their - knowledge about the network through which IP addresses they ask about. - - But with the advent of bridges, Tor clients are asking about IP - addresses that aren't in the main directory. In particular, bridge - users inform the central Vidalia servers about each bridge as they - discover it and their Vidalia tries to map it. - - Also, we wouldn't mind letting Vidalia do a GeoIP lookup on the client's - own IP address, so it can provide a more useful map. - - Finally, Vidalia's central servers leave users open to partitioning - attacks, even if they can't target specific users. Further, as we - start using GeoIP results for more operational or security-relevant - goals, such as avoiding or including particular countries in circuits, - it becomes more important that users can't be singled out in terms of - their IP-to-country mapping beliefs. - -2. The available GeoIP databases - - There are at least two classes of GeoIP database out there: "IP to - country", which tells us the country code for the IP address but - no more details, and "IP to city", which tells us the country code, - the name of the city, and some basic latitude/longitude guesses. - - A recent ip-to-country.csv is 3421362 bytes. Compressed, it is 564252 - bytes. A typical line is: - "205500992","208605279","US","USA","UNITED STATES" - http://ip-to-country.webhosting.info/node/view/5 - - Similarly, the maxmind GeoLite Country database is also about 500KB - compressed. - http://www.maxmind.com/app/geolitecountry - - The maxmind GeoLite City database gives more finegrained detail like - geo coordinates and city name. Vidalia currently makes use of this - information. On the other hand it's 16MB compressed. A typical line is: - 206.124.149.146,Bellevue,WA,US,47.6051,-122.1134 - http://www.maxmind.com/app/geolitecity - - There are other databases out there, like - http://www.hostip.info/faq.html - http://www.webconfs.com/ip-to-city.php - that want more attention, but for now let's assume that all the db's - are around this size. - -3. What we'd like to solve - - Goal #1a: Tor relays collect IP-to-country user stats and publish - sanitized versions. - Goal #1b: Tor bridges collect IP-to-country user stats and publish - sanitized versions. - - Goal #2a: Vidalia learns IP-to-city stats for Tor relays, for better - mapping. - Goal #2b: Vidalia learns IP-to-country stats for Tor relays, so the user - can pick countries for her paths. - - Goal #3: Vidalia doesn't do external lookups on bridge relay addresses. - - Goal #4: Vidalia resolves the Tor client's IP-to-country or IP-to-city - for better mapping. - - Goal #5: Reduce partitioning opportunities where Vidalia central - servers can give different (distinguishing) responses. - -4. Solution overview - - Our goal is to allow Tor relays, bridges, and clients to learn enough - GeoIP information so they can do local private queries. - -4.1. The IP-to-country db - - Directory authorities should publish a "geoip" file that contains - IP-to-country mappings. Directory caches will mirror it, and Tor clients - and relays (including bridge relays) will fetch it. Thus we can solve - goals 1a and 1b (publish sanitized usage info). Controllers could also - use this to solve goal 2b (choosing path by country attributes). It - also solves goal 4 (learning the Tor client's country), though for - huge countries like the US we'd still need to decide where the "middle" - should be when we're mapping that address. - - The IP-to-country details are described further in Sections 5 and - 6 below. - - [RESOLUTION: The geoip file in 0.2.0.x is not distributed through - Tor. Instead, it is shipped with the bundle.] - -4.2. The IP-to-city db - - In an ideal world, the IP-to-city db would be small enough that we - could distribute it in the above manner too. But for now, it is too - large. Here's where the design choice forks. - - Option A: Vidalia should continue doing its anonymized IP-to-city - queries. Thus we can achieve goals 2a and 2b. We would solve goal - 3 by only doing lookups on descriptors that are purpose "general" - (see Section 4.2.1 for how). We would leave goal 5 unsolved. - - Option B: Each directory authority should keep an IP-to-city db, - lookup the value for each router it lists, and include that line in - the router's network-status entry. The network-status consensus would - then use the line that appears in the majority of votes. This approach - also solves goals 2a and 2b, goal 3 (Vidalia doesn't do any lookups - at all now), and goal 5 (reduced partitioning risks). - - Option B has the advantage that Vidalia can simplify its operation, - and the advantage that this consensus IP-to-city data is available to - other controllers besides just Vidalia. But it has the disadvantage - that the networkstatus consensus becomes larger, even though most of - the GeoIP information won't change from one consensus to the next. Is - there another reasonable location for it that can provide similar - consensus security properties? - - [RESOLUTION: IP-to-city is not supported.] - -4.2.1. Controllers can query for router annotations - - Vidalia needs to stop doing queries on bridge relay IP addresses. - It could do that by only doing lookups on descriptors that are in - the networkstatus consensus, but that precludes designs like Blossom - that might want to map its relay locations. The best answer is that it - should learn the router annotations, with a new controller 'getinfo' - command: - "GETINFO desc-annotations/id/" - which would respond with something like - @downloaded-at 2007-11-29 08:06:38 - @source "128.31.0.34" - @purpose bridge - - [We could also make the answer include the digest for the router in - question, which would enable us to ask GETINFO router-annotations/all. - Is this worth it? -RD] - - Then Vidalia can avoid doing lookups on descriptors with purpose - "bridge". Even better would be to add a new annotation "@private true" - so Vidalia can know how to handle new purposes that we haven't created - yet. Vidalia could special-case "bridge" for now, for compatibility - with the current 0.2.0.x-alphas. - -4.3. Recommendation - - My overall recommendation is that we should implement 4.1 soon - (e.g. early in 0.2.1.x), and we can go with 4.2 option A for now, - with the hope that later we discover a better way to distribute the - IP-to-city info and can switch to 4.2 option B. - - Below we discuss more how to go about achieving 4.1. - -5. Publishing and caching the GeoIP (IP-to-country) database - - Each v3 directory authority should put a copy of the "geoip" file in - its datadirectory. Then its network-status votes should include a hash - of this file (Recommended-geoip-hash: %s), and the resulting consensus - directory should specify the consensus hash. - - There should be a new URL for fetching this geoip db (by "current.z" - for testing purposes, and by hash.z for typical downloads). Authorities - should fetch and serve the one listed in the consensus, even when they - vote for their own. This would argue for storing the cached version - in a better filename than "geoip". - - Directory mirrors should keep a copy of this file available via the - same URLs. - - We assume that the file would change at most a few times a month. Should - Tor ship with a bootstrap geoip file? An out-of-date geoip file may - open you up to partitioning attacks, but for the most part it won't - be that different. - - There should be a config option to disable updating the geoip file, - in case users want to use their own file (e.g. they have a proprietary - GeoIP file they prefer to use). In that case we leave it up to the - user to update his geoip file out-of-band. - - [XXX Should consider forward/backward compatibility, e.g. if we want - to move to a new geoip file format. -RD] - - [RESOLUTION: Not done over Tor.] - -6. Controllers use the IP-to-country db for mapping and for path building - - Down the road, Vidalia could use the IP-to-country mappings for placing - on its map: - - The location of the client - - The location of the bridges, or other relays not in the - networkstatus, on the map. - - Any relays that it doesn't yet have an IP-to-city answer for. - - Other controllers can also use it to set EntryNodes, ExitNodes, etc - in a per-country way. - - To support these features, we need to export the IP-to-country data - via the Tor controller protocol. - - Is it sufficient just to add a new GETINFO command? - GETINFO ip-to-country/128.31.0.34 - 250+ip-to-country/128.31.0.34="US","USA","UNITED STATES" - - [RESOLUTION: Not done now, except for the getinfo command.] - -6.1. Other interfaces - - Robert Hogan has also suggested a - - GETINFO relays-by-country/cn - - as well as torrc options for ExitCountryCodes, EntryCountryCodes, - ExcludeCountryCodes, etc. - - [RESOLUTION: Not implemented in 0.2.0.x. Fodder for a future proposal.] - -7. Relays and bridges use the IP-to-country db for usage summaries - - Once bridges have a GeoIP database locally, they can start to publish - sanitized summaries of client usage -- how many users they see and from - what countries. This might also be a more useful way for ordinary Tor - relays to convey the level of usage they see, which would allow us to - switch to using directory guards for all users by default. - - But how to safely summarize this information without opening too many - anonymity leaks? - -7.1 Attacks to think about - - First, note that we need to have a large enough time window that we're - not aiding correlation attacks much. I hope 24 hours is enough. So - that means no publishing stats until you've been up at least 24 hours. - And you can't publish follow-up stats more often than every 24 hours, - or people could look at the differential. - - Second, note that we need to be sufficiently vague about the IP - addresses we're reporting. We are hoping that just specifying the - country will be vague enough. But a) what about active attacks where - we convince a bridge to use a GeoIP db that labels each suspect IP - address as a unique country? We have to assume that the consensus GeoIP - db won't be malicious in this way. And b) could such singling-out - attacks occur naturally, for example because of countries that have - a very small IP space? We should investigate that. - -7.2. Granularity of users - - Do we only want to report countries that have a sufficient anonymity set - (that is, number of users) for the day? For example, we might avoid - listing any countries that have seen less than five addresses over - the 24 hour period. This approach would be helpful in reducing the - singling-out opportunities -- in the extreme case, we could imagine a - situation where one blogger from the Sudan used Tor on a given day, and - we can discover which entry guard she used. - - But I fear that especially for bridges, seeing only one hit from a - given country in a given day may be quite common. - - As a compromise, we should start out with an "Other" category in - the reported stats, which is the sum of unlisted countries; if that - category is consistently interesting, we can think harder about how - to get the right data from it safely. - - But note that bridge summaries will not be made public individually, - since doing so would help people enumerate bridges. Whereas summaries - from normal relays will be public. So perhaps that means we can afford - to be more specific in bridge summaries? In particular, I'm thinking the - "other" category should be used by public relays but not for bridges - (or if it is, used with a lower threshold). - - Even for countries that have many Tor users, we might not want to be - too specific about how many users we've seen. For example, we might - round down the number of users we report to the nearest multiple of 5. - My instinct for now is that this won't be that useful. - -7.3 Other issues - - Another note: we'll likely be overreporting in the case of users with - dynamic IP addresses: if they rotate to a new address over the course - of the day, we'll count them twice. So be it. - -7.4. Where to publish the summaries? - - We designed extrainfo documents for information like this. So they - should just be more entries in the extrainfo doc. - - But if we want to publish summaries every 24 hours (no more often, - no less often), aren't we tried to the router descriptor publishing - schedule? That is, if we publish a new router descriptor at the 18 - hour mark, and nothing much has changed at the 24 hour mark, won't - the new descriptor get dropped as being "cosmetically similar", and - then nobody will know to ask about the new extrainfo document? - - One solution would be to make and remember the 24 hour summary at the - 24 hour mark, but not actually publish it anywhere until we happen to - publish a new descriptor for other reasons. If we happen to go down - before publishing a new descriptor, then so be it, at least we tried. - -7.5. What if the relay is unreachable or goes to sleep? - - Even if you've been up for 24 hours, if you were hibernating for 18 - of them, then we're not getting as much fuzziness as we'd like. So - I guess that means that we need a 24-hour period of being "awake" - before we'll willing to publish a summary. A similar attack works if - you've been awake but unreachable for the first 18 of the 24 hours. As - another example, a bridge that's on a laptop might be suspended for - some of each day. - - This implies that some relays and bridges will never publish summary - stats, because they're not ever reliably working for 24 hours in - a row. If a significant percentage of our reporters end up being in - this boat, we should investigate whether we can accumulate 24 hours of - "usefulness", even if there are holes in the middle, and publish based - on that. - - What other issues are like this? It seems that just moving to a new - IP address shouldn't be a reason to cancel stats publishing, assuming - we were usable at each address. - -7.6. IP addresses that aren't in the geoip db - - Some IP addresses aren't in the public geoip databases. In particular, - I've found that a lot of African countries are missing, but there - are also some common ones in the US that are missing, like parts of - Comcast. We could just lump unknown IP addresses into the "other" - category, but it might be useful to gather a general sense of how many - lookups are failing entirely, by adding a separate "Unknown" category. - - We could also contribute back to the geoip db, by letting bridges set - a config option to report the actual IP addresses that failed their - lookup. Then the bridge authority operators can manually make sure - the correct answer will be in later geoip files. This config option - should be disabled by default. - -7.7 Bringing it all together - - So here's the plan: - - 24 hours after starting up (modulo Section 7.5 above), bridges and - relays should construct a daily summary of client countries they've - seen, including the above "Unknown" category (Section 7.6) as well. - - Non-bridge relays lump all countries with less than K (e.g. K=5) users - into the "Other" category (see Sec 7.2 above), whereas bridge relays are - willing to list a country even when it has only one user for the day. - - Whenever we have a daily summary on record, we include it in our - extrainfo document whenever we publish one. The daily summary we - remember locally gets replaced with a newer one when another 24 - hours pass. - -7.8. Some forward secrecy - - How should we remember addresses locally? If we convert them into - country-codes immediately, we will count them again if we see them - again. On the other hand, we don't really want to keep a list hanging - around of all IP addresses we've seen in the past 24 hours. - - Step one is that we should never write this stuff to disk. Keeping it - only in ram will make things somewhat better. Step two is to avoid - keeping any timestamps associated with it: rather than a rolling - 24-hour window, which would require us to remember the various times - we've seen that address, we can instead just throw out the whole list - every 24 hours and start over. - - We could hash the addresses, and then compare hashes when deciding if - we've seen a given address before. We could even do keyed hashes. Or - Bloom filters. But if our goal is to defend against an adversary - who steals a copy of our ram while we're running and then does - guess-and-check on whatever blob we're keeping, we're in bad shape. - - We could drop the last octet of the IP address as soon as we see - it. That would cause us to undercount some users from cablemodem and - DSL networks that have a high density of Tor users. And it wouldn't - really help that much -- indeed, the extent to which it does help is - exactly the extent to which it makes our stats less useful. - - Other ideas? - diff --git a/doc/spec/proposals/127-dirport-mirrors-downloads.txt b/doc/spec/proposals/127-dirport-mirrors-downloads.txt deleted file mode 100644 index 1b55a02d6..000000000 --- a/doc/spec/proposals/127-dirport-mirrors-downloads.txt +++ /dev/null @@ -1,157 +0,0 @@ -Filename: 127-dirport-mirrors-downloads.txt -Title: Relaying dirport requests to Tor download site / website -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 2007-12-02 -Status: Draft - -1. Overview - - Some countries and networks block connections to the Tor website. As - time goes by, this will remain a problem and it may even become worse. - - We have a big pile of mirrors (google for "Tor mirrors"), but few of - our users think to try a search like that. Also, many of these mirrors - might be automatically blocked since their pages contain words that - might cause them to get banned. And lastly, we can imagine a future - where the blockers are aware of the mirror list too. - - Here we describe a new set of URLs for Tor's DirPort that will relay - connections from users to the official Tor download site. Rather than - trying to cache a bunch of new Tor packages (which is a hassle in terms - of keeping them up to date, and a hassle in terms of drive space used), - we instead just proxy the requests directly to Tor's /dist page. - - Specifically, we should support - - GET /tor/dist/$1 - - and - - GET /tor/website/$1 - -2. Direct connections, one-hop circuits, or three-hop circuits? - - We could relay the connections directly to the download site -- but - this produces recognizable outgoing traffic on the bridge or cache's - network, which will probably surprise our nice volunteers. (Is this - a good enough reason to discard the direct connection idea?) - - Even if we don't do direct connections, should we do a one-hop - begindir-style connection to the mirror site (make a one-hop circuit - to it, then send a 'begindir' cell down the circuit), or should we do - a normal three-hop anonymized connection? - - If these mirrors are mainly bridges, doing either a direct or a one-hop - connection creates another way to enumerate bridges. That would argue - for three-hop. On the other hand, downloading a 10+ megabyte installer - through a normal Tor circuit can't be fun. But if you're already getting - throttled a lot because you're in the "relayed traffic" bucket, you're - going to have to accept a slow transfer anyway. So three-hop it is. - - Speaking of which, we would want to label this connection - as "relay" traffic for the purposes of rate limiting; see - connection_counts_as_relayed_traffic() and or_conn->client_used. This - will be a bit tricky though, because these connections will use the - bridge's guards. - -3. Scanning resistance - - One other goal we'd like to achieve, or at least not hinder, is making - it hard to scan large swaths of the Internet to look for responses - that indicate a bridge. - - In general this is a really hard problem, so we shouldn't demand to - solve it here. But we can note that some bridges should open their - DirPort (and offer this functionality), and others shouldn't. Then - some bridges provide a download mirror while others can remain - scanning-resistant. - -4. Integrity checking - - If we serve this stuff in plaintext from the bridge, anybody in between - the user and the bridge can intercept and modify it. The bridge can too. - - If we do an anonymized three-hop connection, the exit node can also - intercept and modify the exe it sends back. - - Are we setting ourselves up for rogue exit relays, or rogue bridges, - that trojan our users? - - Answer #1: Users need to do pgp signature checking. Not a very good - answer, a) because it's complex, and b) because they don't know the - right signing keys in the first place. - - Answer #2: The mirrors could exit from a specific Tor relay, using the - '.exit' notation. This would make connections a bit more brittle, but - would resolve the rogue exit relay issue. We could even round-robin - among several, and the list could be dynamic -- for example, all the - relays with an Authority flag that allow exits to the Tor website. - - Answer #3: The mirrors should connect to the main distribution site - via SSL. That way the exit relay can't influence anything. - - Answer #4: We could suggest that users only use trusted bridges for - fetching a copy of Tor. Hopefully they heard about the bridge from a - trusted source rather than from the adversary. - - Answer #5: What if the adversary is trawling for Tor downloads by - network signature -- either by looking for known bytes in the binary, - or by looking for "GET /tor/dist/"? It would be nice to encrypt the - connection from the bridge user to the bridge. And we can! The bridge - already supports TLS. Rather than initiating a TLS renegotiation after - connecting to the ORPort, the user should actually request a URL. Then - the ORPort can either pass the connection off as a linked conn to the - dirport, or renegotiate and become a Tor connection, depending on how - the client behaves. - -5. Linked connections: at what level should we proxy? - - Check out the connection_ap_make_link() function, as called from - directory.c. Tor clients use this to create a "fake" socks connection - back to themselves, and then they attach a directory request to it, - so they can launch directory fetches via Tor. We can piggyback on - this feature. - - We need to decide if we're going to be passing the bytes back and - forth between the web browser and the main distribution site, or if - we're going to be actually acting like a proxy (parsing out the file - they want, fetching that file, and serving it back). - - Advantages of proxying without looking inside: - - We don't need to build any sort of http support (including - continues, partial fetches, etc etc). - Disadvantages: - - If the browser thinks it's speaking http, are there easy ways - to pass the bytes to an https server and have everything work - correctly? At the least, it would seem that the browser would - complain about the cert. More generally, ssl wants to be negotiated - before the URL and headers are sent, yet we need to read the URL - and headers to know that this is a mirror request; so we have an - ordering problem here. - - Makes it harder to do caching later on, if we don't look at what - we're relaying. (It might be useful down the road to cache the - answers to popular requests, so we don't have to keep getting - them again.) - -6. Outstanding problems - - 1) HTTP proxies already exist. Why waste our time cloning one - badly? When we clone existing stuff, we usually regret it. - - 2) It's overbroad. We only seem to need a secure get-a-tor feature, - and instead we're contemplating building a locked-down HTTP proxy. - - 3) It's going to add a fair bit of complexity to our code. We do - not currently implement HTTPS. We'd need to refactor lots of the - low-level connection stuff so that "SSL" and "Cell-based" were no - longer synonymous. - - 4) It's still unclear how effective this proposal would be in - practice. You need to know that this feature exists, which means - somebody needs to tell you about a bridge (mirror) address and tell - you how to use it. And if they're doing that, they could (e.g.) tell - you about a gmail autoresponder address just as easily, and then you'd - get better authentication of the Tor program to boot. - diff --git a/doc/spec/proposals/128-bridge-families.txt b/doc/spec/proposals/128-bridge-families.txt deleted file mode 100644 index e8a0050c3..000000000 --- a/doc/spec/proposals/128-bridge-families.txt +++ /dev/null @@ -1,66 +0,0 @@ -Filename: 128-bridge-families.txt -Title: Families of private bridges -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 2007-12-xx -Status: Dead - -1. Overview - - Proposal 125 introduced the basic notion of how bridge authorities, - bridge relays, and bridge users should behave. But it doesn't get into - the various mechanisms of how to distribute bridge relay addresses to - bridge users. - - One of the mechanisms we have in mind is called 'families of bridges'. - If a bridge user knows about only one private bridge, and that bridge - shuts off for the night or gets a new dynamic IP address, the bridge - user is out of luck and needs to re-bootstrap manually or wait and - hope it comes back. On the other hand, if the bridge user knows about - a family of bridges, then as long as one of those bridges is still - reachable his Tor client can automatically learn about where the - other bridges have gone. - - So in this design, a single volunteer could run multiple coordinated - bridges, or a group of volunteers could each run a bridge. We abstract - out the details of how these volunteers find each other and decide to - set up a family. - -2. Other notes. - - somebody needs to run a bridge authority - - it needs to have a torrc option to publish networkstatuses of its bridges - - it should also do reachability testing just of those bridges - - people ask for the bridge networkstatus by asking for a url that - contains a password. (it's safe to do this because of begin_dir.) - - so the bridge users need to know a) a password, and b) a bridge - authority line. - - the bridge users need to know the bridge authority line. - - the bridge authority needs to know the password. - -3. Current state - - I implemented a BridgePassword config option. Bridge authorities - should set it, and users who want to use those bridge authorities - should set it. - - Now there is a new directory URL "/tor/networkstatus-bridges" that - directory mirrors serve if BridgeAuthoritativeDir is set and it's a - begin_dir connection. It looks for the header - Authorization: Basic %s - where %s is the base-64 bridge password. - - I never got around to teaching clients how to set the header though, - so it may or may not, and may or may not do what we ultimate want. - - I've marked this proposal dead; it really never should have left the - ideas/ directory. Somebody should pick it up sometime and finish the - design and implementation. - diff --git a/doc/spec/proposals/129-reject-plaintext-ports.txt b/doc/spec/proposals/129-reject-plaintext-ports.txt deleted file mode 100644 index d4767d03d..000000000 --- a/doc/spec/proposals/129-reject-plaintext-ports.txt +++ /dev/null @@ -1,116 +0,0 @@ -Filename: 129-reject-plaintext-ports.txt -Title: Block Insecure Protocols by Default -Version: $Revision$ -Last-Modified: $Date$ -Author: Kevin Bauer & Damon McCoy -Created: 2008-01-15 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - Below is a proposal to mitigate insecure protocol use over Tor. - - This document 1) demonstrates the extent to which insecure protocols are - currently used within the Tor network, and 2) proposes a simple solution - to prevent users from unknowingly using these insecure protocols. By - insecure, we consider protocols that explicitly leak sensitive user names - and/or passwords, such as POP, IMAP, Telnet, and FTP. - -Motivation: - - As part of a general study of Tor use in 2006/2007 [1], we attempted to - understand what types of protocols are used over Tor. While we observed a - enormous volume of Web and Peer-to-peer traffic, we were surprised by the - number of insecure protocols that were used over Tor. For example, over an - 8 day observation period, we observed the following number of connections - over insecure protocols: - - POP and IMAP:10,326 connections - Telnet: 8,401 connections - FTP: 3,788 connections - - Each of the above listed protocols exchange user name and password - information in plain-text. As an upper bound, we could have observed - 22,515 user names and passwords. This observation echos the reports of - a Tor router logging and posting e-mail passwords in August 2007 [2]. The - response from the Tor community has been to further educate users - about the dangers of using insecure protocols over Tor. However, we - recently repeated our Tor usage study from last year and noticed that the - trend in insecure protocol use has not declined. Therefore, we propose that - additional steps be taken to protect naive Tor users from inadvertently - exposing their identities (and even passwords) over Tor. - -Security Implications: - - This proposal is intended to improve Tor's security by limiting the - use of insecure protocols. - - Roger added: By adding these warnings for only some of the risky - behavior, users may do other risky behavior, not get a warning, and - believe that it is therefore safe. But overall, I think it's better - to warn for some of it than to warn for none of it. - -Specification: - - As an initial step towards mitigating the use of the above-mentioned - insecure protocols, we propose that the default ports for each respective - insecure service be blocked at the Tor client's socks proxy. These default - ports include: - - 23 - Telnet - 109 - POP2 - 110 - POP3 - 143 - IMAP - - Notice that FTP is not included in the proposed list of ports to block. This - is because FTP is often used anonymously, i.e., without any identifying - user name or password. - - This blocking scheme can be implemented as a set of flags in the client's - torrc configuration file: - - BlockInsecureProtocols 0|1 - WarnInsecureProtocols 0|1 - - When the warning flag is activated, a message should be displayed to - the user similar to the message given when Tor's socks proxy is given an IP - address rather than resolving a host name. - - We recommend that the default torrc configuration file block insecure - protocols and provide a warning to the user to explain the behavior. - - Finally, there are many popular web pages that do not offer secure - login features, such as MySpace, and it would be prudent to provide - additional rules to Privoxy to attempt to protect users from unknowingly - submitting their login credentials in plain-text. - -Compatibility: - - None, as the proposed changes are to be implemented in the client. - -References: - - [1] Shining Light in Dark Places: A Study of Anonymous Network Usage. - University of Colorado Technical Report CU-CS-1032-07. August 2007. - - [2] Rogue Nodes Turn Tor Anonymizer Into Eavesdropper's Paradise. - http://www.wired.com/politics/security/news/2007/09/embassy_hacks. - Wired. September 10, 2007. - -Implementation: - - Roger added this feature in - http://archives.seul.org/or/cvs/Jan-2008/msg00182.html - He also added a status event for Vidalia to recognize attempts to use - vulnerable-plaintext ports, so it can help the user understand what's - going on and how to fix it. - -Next steps: - - a) Vidalia should learn to recognize this controller status event, - so we don't leave users out in the cold when we enable this feature. - - b) We should decide which ports to reject by default. The current - consensus is 23,109,110,143 -- the same set that we warn for now. - diff --git a/doc/spec/proposals/130-v2-conn-protocol.txt b/doc/spec/proposals/130-v2-conn-protocol.txt deleted file mode 100644 index 16f5bf284..000000000 --- a/doc/spec/proposals/130-v2-conn-protocol.txt +++ /dev/null @@ -1,186 +0,0 @@ -Filename: 130-v2-conn-protocol.txt -Title: Version 2 Tor connection protocol -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 2007-10-25 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - This proposal describes the significant changes to be made in the v2 - Tor connection protocol. - - This proposal relates to other proposals as follows: - - It refers to and supersedes: - Proposal 124: Blocking resistant TLS certificate usage - It refers to aspects of: - Proposal 105: Version negotiation for the Tor protocol - - - In summary, The Tor connection protocol has been in need of a redesign - for a while. This proposal describes how we can add to the Tor - protocol: - - - A new TLS handshake (to achieve blocking resistance without - breaking backward compatibility) - - Version negotiation (so that future connection protocol changes - can happen without breaking compatibility) - - The actual changes in the v2 Tor connection protocol. - -Motivation: - - For motivation, see proposal 124. - -Proposal: - -0. Terminology - - The version of the Tor connection protocol implemented up to now is - "version 1". This proposal describes "version 2". - - "Old" or "Older" versions of Tor are ones not aware that version 2 - of this protocol exists; - "New" or "Newer" versions are ones that are. - - The connection initiator is referred to below as the Client; the - connection responder is referred to below as the Server. - -1. The revised TLS handshake. - - For motivation, see proposal 124. This is a simplified version of the - handshake that uses TLS's renegotiation capability in order to avoid - some of the extraneous steps in proposal 124. - - The Client connects to the Server and, as in ordinary TLS, sends a - list of ciphers. Older versions of Tor will send only ciphers from - the list: - TLS_DHE_RSA_WITH_AES_256_CBC_SHA - TLS_DHE_RSA_WITH_AES_128_CBC_SHA - SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA - SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA - Clients that support the revised handshake will send the recommended - list of ciphers from proposal 124, in order to emulate the behavior of - a web browser. - - If the server notices that the list of ciphers contains only ciphers - from this list, it proceeds with Tor's version 1 TLS handshake as - documented in tor-spec.txt. - - (The server may also notice cipher lists used by other implementations - of the Tor protocol (in particular, the BouncyCastle default cipher - list as used by some Java-based implementations), and whitelist them.) - - On the other hand, if the server sees a list of ciphers that could not - have been sent from an older implementation (because it includes other - ciphers, and does not match any known-old list), the server sends a - reply containing a single connection certificate, constructed as for - the link certificate in the v1 Tor protocol. The subject names in - this certificate SHOULD NOT have any strings to identify them as - coming from a Tor server. The server does not ask the client for - certificates. - - Old Servers will (mostly) ignore the cipher list and respond as in the v1 - protocol, sending back a two-certificate chain. - - After the Client gets a response from the server, it checks for the - number of certificates it received. If there are two certificates, - the client assumes a V1 connection and proceeds as in tor-spec.txt. - But if there is only one certificate, the client assumes a V2 or later - protocol and continues. - - At this point, the client has established a TLS connection with the - server, but the parties have not been authenticated: the server hasn't - sent its identity certificate, and the client hasn't sent any - certificates at all. To fix this, the client begins a TLS session - renegotiation. This time, the server continues with two certificates - as usual, and asks for certificates so that the client will send - certificates of its own. Because the TLS connection has been - established, all of this is encrypted. (The certificate sent by the - server in the renegotiated connection need not be the same that - as sentin the original connection.) - - The server MUST NOT write any data until the client has renegotiated. - - Once the renegotiation is finished, the server and client check one - another's certificates as in V1. Now they are mutually authenticated. - -1.1. Revised TLS handshake: implementation notes. - - It isn't so easy to adjust server behavior based on the client's - ciphersuite list. Here's how we can do it using OpenSSL. This is a - bit of an abuse of the OpenSSL APIs, but it's the best we can do, and - we won't have to do it forever. - - We can use OpenSSL's SSL_set_info_callback() to register a function to - be called when the state changes. The type/state tuple of - SSL_CB_ACCEPT_LOOP/SSL3_ST_SW_SRVR_HELLO_A - happens when we have completely parsed the client hello, and are about - to send a response. From this callback, we can check the cipherlist - and act accordingly: - - * If the ciphersuite list indicates a v1 protocol, we set the - verify mode to SSL_VERIFY_NONE with a callback (so we get - certificates). - - * If the ciphersuite list indicates a v2 protocol, we set the - verify mode to SSL_VERIFY_NONE with no callback (so we get - no certificates) and set the SSL_MODE_NO_AUTO_CHAIN flag (so that - we send only 1 certificate in the response. - - Once the handshake is done, the server clears the - SSL_MODE_NO_AUTO_CHAIN flag and sets the callback as for the V1 - protocol. It then starts reading. - - The other problem to take care of is missing ciphers and OpenSSL's - cipher sorting algorithms. The two main issues are a) OpenSSL doesn't - support some of the default ciphers that Firefox advertises, and b) - OpenSSL sorts the list of ciphers it offers in a different way than - Firefox sorts them, so unless we fix that Tor will still look different - than Firefox. - [XXXX more on this.] - - -1.2. Compatibility for clients using libraries less hackable than OpenSSL. - - As discussed in proposal 105, servers advertise which protocol - versions they support in their router descriptors. Clients can simply - behave as v1 clients when connecting to servers that do not support - link version 2 or higher, and as v2 clients when connecting to servers - that do support link version 2 or higher. - - (Servers can't use this strategy because we do not assume that servers - know one another's capabilities when connecting.) - -2. Version negotiation. - - Version negotiation proceeds as described in proposal 105, except as - follows: - - * Version negotiation only happens if the TLS handshake as described - above completes. - - * The TLS renegotiation must be finished before the client sends a - VERSIONS cell; the server sends its VERSIONS cell in response. - - * The VERSIONS cell uses the following variable-width format: - Circuit [2 octets; set to 0] - Command [1 octet; set to 7 for VERSIONS] - Length [2 octets; big-endian] - Data [Length bytes] - - The Data in the cell is a series of big-endian two-byte integers. - - * It is not allowed to negotiate V1 conections once the v2 protocol - has been used. If this happens, Tor instances should close the - connection. - -3. The rest of the "v2" protocol - - Once a v2 protocol has been negotiated, NETINFO cells are exchanged - as in proposal 105, and communications begin as per tor-spec.txt. - Until NETINFO cells have been exchanged, the connection is not open. - - diff --git a/doc/spec/proposals/131-verify-tor-usage.txt b/doc/spec/proposals/131-verify-tor-usage.txt deleted file mode 100644 index 268713918..000000000 --- a/doc/spec/proposals/131-verify-tor-usage.txt +++ /dev/null @@ -1,150 +0,0 @@ -Filename: 131-verify-tor-usage.txt -Title: Help users to verify they are using Tor -Version: $Revision$ -Last-Modified: $Date$ -Author: Steven J. Murdoch -Created: 2008-01-25 -Status: Needs-Revision - -Overview: - - Websites for checking whether a user is accessing them via Tor are a - very helpful aid to configuring web browsers correctly. Existing - solutions have both false positives and false negatives when - checking if Tor is being used. This proposal will discuss how to - modify Tor so as to make testing more reliable. - -Motivation: - - Currently deployed websites for detecting Tor use work by comparing - the client IP address for a request with a list of known Tor nodes. - This approach is generally effective, but suffers from both false - positives and false negatives. - - If a user has a Tor exit node installed, or just happens to have - been allocated an IP address previously used by a Tor exit node, any - web requests will be incorrectly flagged as coming from Tor. If any - customer of an ISP which implements a transparent proxy runs an exit - node, all other users of the ISP will be flagged as Tor users. - - Conversely, if the exit node chosen by a Tor user has not yet been - recorded by the Tor checking website, requests will be incorrectly - flagged as not coming via Tor. - - The only reliable way to tell whether Tor is being used or not is for - the Tor client to flag this to the browser. - -Proposal: - - A DNS name should be registered and point to an IP address - controlled by the Tor project and likely to remain so for the - useful lifetime of a Tor client. A web server should be placed - at this IP address. - - Tor should be modified to treat requests to port 80, at the - specified DNS name or IP address specially. Instead of opening a - circuit, it should respond to a HTTP request with a helpful web - page: - - - If the request to open a connection was to the domain name, the web - page should state that Tor is working properly. - - If the request was to the IP address, the web page should state - that there is a DNS-leakage vulnerability. - - If the request goes through to the real web server, the page - should state that Tor has not been set up properly. - -Extensions: - - Identifying proxy server: - - If needed, other applications between the web browser and Tor (e.g. - Polipo and Privoxy) could piggyback on the same mechanism to flag - whether they are in use. All three possible web pages should include - a machine-readable placeholder, into which another program could - insert their own message. - - For example, the webpage returned by Tor to indicate a successful - configuration could include the following HTML: -

Connection chain

-
    -
  • Tor 0.1.2.14-alpha
    • - -
- - When the proxy server observes this string, in response to a request - for the Tor connectivity check web page, it would prepend it's own - message, resulting in the following being returned to the web - browser: -

Connection chain -
    -
  • Tor 0.1.2.14-alpha
    • -
  • Polipo version 1.0.4
    • - -
- - Checking external connectivity: - - If Tor intercepts a request, and returns a response itself, the user - will not actually confirm whether Tor is able to build a successful - circuit. It may then be advantageous to include an image in the web - page which is loaded from a different domain. If this is able to be - loaded then the user will know that external connectivity through - Tor works. - - Automatic Firefox Notification: - - All forms of the website should return valid XHTML and have a - hidden link with an id attribute "TorCheckResult" and a target - property that can be queried to determine the result. For example, - a hidden link would convey success like this: - - - - failure like this: - - - - and DNS leaks like this: - - - - Firefox extensions such as Torbutton would then be able to - issue an XMLHttpRequest for the page and query the result - with resultXML.getElementById("TorCheckResult").target - to automatically report the Tor status to the user when - they first attempt to enable Tor activity, or whenever - they request a check from the extension preferences window. - - If the check website is to be themed with heavy graphics and/or - extensive documentation, the check result itself should be - contained in a seperate lightweight iframe that extensions can - request via an alternate url. - -Security and resiliency implications: - - What attacks are possible? - - If the IP address used for this feature moves there will be two - consequences: - - A new website at this IP address will remain inaccessible over - Tor - - Tor users who are leaking DNS will be informed that Tor is not - working, rather than that it is active but leaking DNS - We should thus attempt to find an IP address which we reasonably - believe can remain static. - -Open issues: - - If a Tor version which does not support this extra feature is used, - the webpage returned will indicate that Tor is not being used. Can - this be safely fixed? - -Related work: - - The proposed mechanism is very similar to config.privoxy.org. The - most significant difference is that if the web browser is - misconfigured, Tor will only get an IP address. Even in this case, - Tor should be able to respond with a webpage to notify the user of how - to fix the problem. This also implies that Tor must be told of the - special IP address, and so must be effectively permanent. diff --git a/doc/spec/proposals/132-browser-check-tor-service.txt b/doc/spec/proposals/132-browser-check-tor-service.txt deleted file mode 100644 index d07a10dcd..000000000 --- a/doc/spec/proposals/132-browser-check-tor-service.txt +++ /dev/null @@ -1,147 +0,0 @@ -Filename: 132-browser-check-tor-service.txt -Title: A Tor Web Service For Verifying Correct Browser Configuration -Version: $Revision$ -Last-Modified: $Date$ -Author: Robert Hogan -Created: 2008-03-08 -Status: Draft - -Overview: - - Tor should operate a primitive web service on the loopback network device - that tests the operation of user's browser, privacy proxy and Tor client. - The tests are performed by serving unique, randomly generated elements in - image URLs embedded in static HTML. The images are only displayed if the DNS - and HTTP requests for them are routed through Tor, otherwise the 'alt' text - may be displayed. The proposal assumes that 'alt' text is not displayed on - all browsers so suggests that text and links should accompany each image - advising the user on next steps in case the test fails. - - The service is primarily for the use of controllers, since presumably users - aren't going to want to edit text files and then type something exotic like - 127.0.0.1:9999 into their address bar. In the main use case the controller - will have configured the actual port for the webservice so will know where - to direct the request. It would also be the responsibility of the controller - to ensure the webservice is available, and tor is running, before allowing - the user to access the page through their browser. - -Motivation: - - This is a complementary approach to proposal 131. It overcomes some of the - limitations of the approach described in proposal 131: reliance - on a permanent, real IP address and compatibility with older versions of - Tor. Unlike 131, it is not as useful to Tor users who are not running a - controller. - -Objective: - - Provide a reliable means of helping users to determine if their Tor - installation, privacy proxy and browser are properly configured for - anonymous browsing. - -Proposal: - - When configured to do so, Tor should run a basic web service available - on a configured port on 127.0.0.1. The purpose of this web service is to - serve a number of basic test images that will allow the user to determine - if their browser is properly configured and that Tor is working normally. - - The service can consist of a single web page with two columns. The left - column contains images, the right column contains advice on what the - display/non-display of the column means. - - The rest of this proposal assumes that the service is running on port - 9999. The port should be configurable, and configuring the port enables the - service. The service must run on 127.0.0.1. - - In all the examples below [uniquesessionid] refers to a random, base64 - encoded string that is unique to the URL it is contained in. Tor only ever - stores the most recently generated [uniquesessionid] for each URL, storing 3 - in total. Tor should generate a [uniquesessionid] for each of the test URLs - below every time a HTTP GET is received at 127.0.0.1:9999 for index.htm. - - The most suitable image for each test case is an implementation decision. - Tor will need to store and serve images for the first and second test - images, and possibly the third (see 'Open Issues'). - - 1. DNS Request Test Image - - This is a HTML element embedded in the page served by Tor at - http://127.0.0.1:9999: - - If you can see - this text, your browser's DNS requests are not being routed through Tor. - - If the browser's DNS request for [uniquesessionid] is routed through Tor, - Tor will intercept the request and return 127.0.0.1 as the resolved IP - address. This will shortly be followed by a HTTP request from the browser - for http://127.0.0.1:9999/torlogo.jpg. This request should be served with - the appropriate image. - - If the browser's DNS request for [uniquesessionid] is not routed through Tor - the browser may display the 'alt' text specified in the html element. The - HTML served by Tor should also contain text accompanying the image to advise - users what it means if they do not see an image. It should also provide a - link to click that provides information on how to remedy the problem. This - behaviour also applies to the images described in 2. and 3. below, so should - be assumed there as well. - - - 2. Proxy Configuration Test Image - - This is a HTML element embedded in the page served by Tor at - http://127.0.0.1:9999: - - If you can see - this text, your browser is not configured to work with Tor. - - If the HTTP request for the resource [uniquesessionid].jpg is received by - Tor it will serve the appropriate image in response. It should serve this - image itself, without attempting to retrieve anything from the Internet. - - If Tor can identify the name of the proxy application requesting the - resource then it could store and serve an image identifying the proxy to the - user. - - 3. Tor Connectivity Test Image - - This is a HTML element embedded in the page served by Tor at - http://127.0.0.1:9999: - - If you - can see this text, your Tor installation cannot connect to the Internet. - - The referenced image should actually exist on the Tor project website. If - Tor receives the request for the above resource it should remove the random - base64 encoded digest from the request (i.e. [uniquesessionid]-) and attempt - to retrieve the real image. - - Even on a fully operational Tor client this test may not always succeed. The - user should be advised that one or more attempts to retrieve this image may - be necessary to confirm a genuine problem. - -Open Issues: - - The final connectivity test relies on an externally maintained resource, if - this resource becomes unavailable the connectivity test will always fail. - Either the text accompanying the test should advise of this possibility or - Tor clients should be advised of the location of the test resource in the - main network directory listings. - - Any number of misconfigurations may make the web service unreachable, it is - the responsibility of the user's controller to recognize these and assist - the user in eliminating them. Tor can mitigate against the specific - misconfiguration of routing HTTP traffic to 127.0.0.1 to Tor itself by - serving such requests through the SOCKS port as well as the configured web - service report. - - Now Tor is inspecting the URLs requested on its SOCKS port and 'dropping' - them. It already inspects for raw IP addresses (to warn of DNS leaks) but - maybe the behaviour proposed here is qualitatively different. Maybe this is - an unwelcome precedent that can be used to beat the project over the head in - future. Or maybe it's not such a bad thing, Tor is merely attempting to make - normally invalid resource requests valid for a given purpose. - diff --git a/doc/spec/proposals/133-unreachable-ors.txt b/doc/spec/proposals/133-unreachable-ors.txt deleted file mode 100644 index a1c2dd854..000000000 --- a/doc/spec/proposals/133-unreachable-ors.txt +++ /dev/null @@ -1,128 +0,0 @@ -Filename: 133-unreachable-ors.txt -Title: Incorporate Unreachable ORs into the Tor Network -Author: Robert Hogan -Created: 2008-03-08 -Status: Draft - -Overview: - - Propose a scheme for harnessing the bandwidth of ORs who cannot currently - participate in the Tor network because they can only make outbound - TCP connections. - -Motivation: - - Restrictive local and remote firewalls are preventing many willing - candidates from becoming ORs on the Tor network.These - ORs have a casual interest in joining the network but their operator is not - sufficiently motivated or adept to complete the necessary router or firewall - configuration. The Tor network is losing out on their bandwidth. At the - moment we don't even know how many such 'candidate' ORs there are. - - -Objective: - - 1. Establish how many ORs are unable to qualify for publication because - they cannot establish that their ORPort is reachable. - - 2. Devise a method for making such ORs available to clients for circuit - building without prejudicing their anonymity. - -Proposal: - - ORs whose ORPort reachability testing fails a specified number of - consecutive times should: - 1. Enlist themselves with the authorities setting a 'Fallback' flag. This - flag indicates that the OR is up and running but cannot connect to - itself. - 2. Open an orconn with all ORs whose fingerprint begins with the same - byte as their own. The management of this orconn will be transferred - entirely to the OR at the other end. - 2. The fallback OR should update it's router status to contain the - 'Running' flag if it has managed to open an orconn with 3/4 of the ORs - with an FP beginning with the same byte as its own. - - Tor ORs who are contacted by fallback ORs requesting an orconn should: - 1. Accept the orconn until they have reached a defined limit of orconn - connections with fallback ORs. - 2. Should only accept such orconn requests from listed fallback ORs who - have an FP beginning with the same byte as its own. - - Tor clients can include fallback ORs in the network by doing the - following: - 1. When building a circuit, observe the fingerprint of each node they - wish to connect to. - 2. When randomly selecting a node from the set of all eligible nodes, - add all published, running fallback nodes to the set where the first - byte of the fingerprint matches the previous node in the circuit. - -Anonymity Implications: - - At least some, and possibly all, nodes on the network will have a set - of nodes that only they and a few others can build circuits on. - - 1. This means that fallback ORs might be unsuitable for use as middlemen - nodes, because if the exit node is the attacker it knows that the - number of nodes that could be the entry guard in the circuit is - reduced to roughly 1/256th of the network, or worse 1/256th of all - nodes listed as Guards. For the same reason, fallback nodes would - appear to be unsuitable for two-hop circuits. - - 2. This is not a problem if fallback ORs are always exit nodes. If - the fallback OR is an attacker it will not be able to reduce the - set of possible nodes for the entry guard any further than a normal, - published OR. - -Possible Attacks/Open Issues: - - 1. Gaming Node Selection - Does running a fallback OR customized for a specific set of published ORs - improve an attacker's chances of seeing traffic from that set of published - ORs? Would such a strategy be any more effective than running published - ORs with other 'attractive' properties? - - 2. DOS Attack - An attacker could prevent all other legitimate fallback ORs with a - given byte-1 in their FP from functioning by running 20 or 30 fallback ORs - and monopolizing all available fallback slots on the published ORs. - This same attacker would then be in a position to monopolize all the - traffic of the fallback ORs on that byte-1 network segment. I'm not sure - what this would allow such an attacker to do. - - 4. Circuit-Sniffing - An observer watching exit traffic from a fallback server will know that the - previous node in the circuit is one of a very small, identifiable - subset of the total ORs in the network. To establish the full path of the - circuit they would only have to watch the exit traffic from the fallback - OR and all the traffic from the 20 or 30 ORs it is likely to be connected - to. This means it is substantially easier to establish all members of a - circuit which has a fallback OR as an exit (sniff and analyse 10-50 (i.e. - 1/256 varying) + 1 ORs) rather than a normal published OR (sniff all 2560 - or so ORs on the network). The same mechanism that allows the client to - expect a specific fallback OR to be available from a specific published OR - allows an attacker to prepare his ground. - - Mitigant: - In terms of the resources and access required to monitor 2000 to 3000 - nodes, the effort of the adversary is not significantly diminished when he - is only interested in 20 or 30. It is hard to see how an adversary who can - obtain access to a randomly selected portion of the Tor network would face - any new or qualitatively different obstacles in attempting to access much - of the rest of it. - - -Implementation Issues: - - The number of ORs this proposal would add to the Tor network is not known. - This is because there is no mechanism at present for recording unsuccessful - attempts to become an OR. If the proposal is considered promising it may be - worthwhile to issue an alpha series release where candidate ORs post a - primitive fallback descriptor to the authority directories. This fallback - descriptor would not contain any other flag that would make it eligible for - selection by clients. It would act solely as a means of sizing the number of - Tor instances that try and fail to become ORs. - - The upper limit on the number of orconns from fallback ORs a normal, - published OR should be willing to accept is an open question. Is one - hundred, mostly idle, such orconns too onerous? - diff --git a/doc/spec/proposals/134-robust-voting.txt b/doc/spec/proposals/134-robust-voting.txt deleted file mode 100644 index 5d5e77fa3..000000000 --- a/doc/spec/proposals/134-robust-voting.txt +++ /dev/null @@ -1,105 +0,0 @@ -Filename: 134-robust-voting.txt -Title: More robust consensus voting with diverse authority sets -Author: Peter Palfrader -Created: 2008-04-01 -Status: Accepted -Target: 0.2.2.x - -Overview: - - A means to arrive at a valid directory consensus even when voters - disagree on who is an authority. - - -Motivation: - - Right now there are about five authoritative directory servers in the - Tor network, tho this number is expected to rise to about 15 eventually. - - Adding a new authority requires synchronized action from all operators of - directory authorities so that at any time during the update at least half of - all authorities are running and agree on who is an authority. The latter - requirement is there so that the authorities can arrive at a common - consensus: Each authority builds the consensus based on the votes from - all authorities it recognizes, and so a different set of recognized - authorities will lead to a different consensus document. - - -Objective: - - The modified voting procedure outlined in this proposal obsoletes the - requirement for most authorities to exactly agree on the list of - authorities. - - -Proposal: - - The vote document each authority generates contains a list of - authorities recognized by the generating authority. This will be - a list of authority identity fingerprints. - - Authorities will accept votes from and serve/mirror votes also for - authorities they do not recognize. (Votes contain the signing, - authority key, and the certificate linking them so they can be - verified even without knowing the authority beforehand.) - - Before building the consensus we will check which votes to use for - building: - - 1) We build a directed graph of which authority/vote recognizes - whom. - 2) (Parts of the graph that aren't reachable, directly or - indirectly, from any authorities we recognize can be discarded - immediately.) - 3) We find the largest fully connected subgraph. - (Should there be more than one subgraph of the same size there - needs to be some arbitrary ordering so we always pick the same. - E.g. pick the one who has the smaller (XOR of all votes' digests) - or something.) - 4) If we are part of that subgraph, great. This is the list of - votes we build our consensus with. - 5) If we are not part of that subgraph, remove all the nodes that - are part of it and go to 3. - - Using this procedure authorities that are updated to recognize a - new authority will continue voting with the old group until a - sufficient number has been updated to arrive at a consensus with - the recently added authority. - - In fact, the old set of authorities will probably be voting among - themselves until all but one has been updated to recognize the - new authority. Then which set of votes is used for consensus - building depends on which of the two equally large sets gets - ordered before the other in step (3) above. - - It is necessary to continue with the process in (5) even if we - are not in the largest subgraph. Otherwise one rogue authority - could create a number of extra votes (by new authorities) so that - everybody stops at 5 and no consensus is built, even tho it would - be trusted by all clients. - - -Anonymity Implications: - - The author does not believe this proposal to have anonymity - implications. - - -Possible Attacks/Open Issues/Some thinking required: - - Q: Can a number (less or exactly half) of the authorities cause an honest - authority to vote for "their" consensus rather than the one that would - result were all authorities taken into account? - - - Q: Can a set of votes from external authorities, i.e of whom we trust either - none or at least not all, cause us to change the set of consensus makers we - pick? - A: Yes, if other authorities decide they rather build a consensus with them - then they'll be thrown out in step 3. But that's ok since those other - authorities will never vote with us anyway. - If we trust none of them then we throw them out even sooner, so no harm done. - - Q: Can this ever force us to build a consensus with authorities we do not - recognize? - A: No, we can never build a fully connected set with them in step 3. diff --git a/doc/spec/proposals/135-private-tor-networks.txt b/doc/spec/proposals/135-private-tor-networks.txt deleted file mode 100644 index 131bbb906..000000000 --- a/doc/spec/proposals/135-private-tor-networks.txt +++ /dev/null @@ -1,283 +0,0 @@ -Filename: 135-private-tor-networks.txt -Title: Simplify Configuration of Private Tor Networks -Version: $Revision$ -Last-Modified: $Date$ -Author: Karsten Loesing -Created: 29-Apr-2008 -Status: Closed -Target: 0.2.1.x -Implemented-In: 0.2.1.2-alpha - -Change history: - - 29-Apr-2008 Initial proposal for or-dev - 19-May-2008 Included changes based on comments by Nick to or-dev and - added a section for test cases. - 18-Jun-2008 Changed testing-network-only configuration option names. - -Overview: - - Configuring a private Tor network has become a time-consuming and - error-prone task with the introduction of the v3 directory protocol. In - addition to that, operators of private Tor networks need to set an - increasing number of non-trivial configuration options, and it is hard - to keep FAQ entries describing this task up-to-date. In this proposal we - (1) suggest to (optionally) accelerate timing of the v3 directory voting - process and (2) introduce an umbrella config option specifically aimed at - creating private Tor networks. - -Design: - - 1. Accelerate Timing of v3 Directory Voting Process - - Tor has reasonable defaults for setting up a large, Internet-scale - network with comparably high latencies and possibly wrong server clocks. - However, those defaults are bad when it comes to quickly setting up a - private Tor network for testing, either on a single node or LAN (things - might be different when creating a test network on PlanetLab or - something). Some time constraints should be made configurable for private - networks. The general idea is to accelerate everything that has to do - with propagation of directory information, but nothing else, so that a - private network is available as soon as possible. (As a possible - safeguard, changing these configuration values could be made dependent on - the umbrella configuration option introduced in 2.) - - 1.1. Initial Voting Schedule - - When a v3 directory does not know any consensus, it assumes an initial, - hard-coded VotingInterval of 30 minutes, VoteDelay of 5 minutes, and - DistDelay of 5 minutes. This is important for multiple, simultaneously - restarted directory authorities to meet at a common time and create an - initial consensus. Unfortunately, this means that it may take up to half - an hour (or even more) for a private Tor network to bootstrap. - - We propose to make these three time constants configurable (note that - V3AuthVotingInterval, V3AuthVoteDelay, and V3AuthDistDelay do not have an - effect on the _initial_ voting schedule, but only on the schedule that a - directory authority votes for). This can be achieved by introducing three - new configuration options: TestingV3AuthInitialVotingInterval, - TestingV3AuthInitialVoteDelay, and TestingV3AuthInitialDistDelay. - - As first safeguards, Tor should only accept configuration values for - TestingV3AuthInitialVotingInterval that divide evenly into the default - value of 30 minutes. The effect is that even if people misconfigured - their directory authorities, they would meet at the default values at the - latest. The second safeguard is to allow configuration only when the - umbrella configuration option TestingTorNetwork is set. - - 1.2. Immediately Provide Reachability Information (Running flag) - - The default behavior of a directory authority is to provide the Running - flag only after the authority is available for at least 30 minutes. The - rationale is that before that time, an authority simply cannot deliver - useful information about other running nodes. But for private Tor - networks this may be different. This is currently implemented in the code - as: - - /** If we've been around for less than this amount of time, our - * reachability information is not accurate. */ - #define DIRSERV_TIME_TO_GET_REACHABILITY_INFO (30*60) - - There should be another configuration option - TestingAuthDirTimeToLearnReachability with a default value of 30 minutes - that can be changed when running testing Tor networks, e.g. to 0 minutes. - The configuration value would simply replace the quoted constant. Again, - changing this option could be safeguarded by requiring the umbrella - configuration option TestingTorNetwork to be set. - - 1.3. Reduce Estimated Descriptor Propagation Time - - Tor currently assumes that it takes up to 10 minutes until router - descriptors are propagated from the authorities to directory caches. - This is not very useful for private Tor networks, and we want to be able - to reduce this time, so that clients can download router descriptors in a - timely manner. - - /** Clients don't download any descriptor this recent, since it will - * probably not have propagated to enough caches. */ - #define ESTIMATED_PROPAGATION_TIME (10*60) - - We suggest to introduce a new config option - TestingEstimatedDescriptorPropagationTime which defaults to 10 minutes, - but that can be set to any lower non-negative value, e.g. 0 minutes. The - same safeguards as in 1.2 could be used here, too. - - 2. Umbrella Option for Setting Up Private Tor Networks - - Setting up a private Tor network requires a number of specific settings - that are not required or useful when running Tor in the public Tor - network. Instead of writing down these options in a FAQ entry, there - should be a single configuration option, e.g. TestingTorNetwork, that - changes all required settings at once. Newer Tor versions would keep the - set of configuration options up-to-date. It should still remain possible - to manually overwrite the settings that the umbrella configuration option - affects. - - The following configuration options are set by TestingTorNetwork: - - - ServerDNSAllowBrokenResolvConf 1 - Ignore the situation that private relays are not aware of any name - servers. - - - DirAllowPrivateAddresses 1 - Allow router descriptors containing private IP addresses. - - - EnforceDistinctSubnets 0 - Permit building circuits with relays in the same subnet. - - - AssumeReachable 1 - Omit self-testing for reachability. - - - AuthDirMaxServersPerAddr 0 - - AuthDirMaxServersPerAuthAddr 0 - Permit an unlimited number of nodes on the same IP address. - - - ClientDNSRejectInternalAddresses 0 - Believe in DNS responses resolving to private IP addresses. - - - ExitPolicyRejectPrivate 0 - Allow exiting to private IP addresses. (This one is a matter of - taste---it might be dangerous to make this a default in a private - network, although people setting up private Tor networks should know - what they are doing.) - - - V3AuthVotingInterval 5 minutes - - V3AuthVoteDelay 20 seconds - - V3AuthDistDelay 20 seconds - Accelerate voting schedule after first consensus has been reached. - - - TestingV3AuthInitialVotingInterval 5 minutes - - TestingV3AuthInitialVoteDelay 20 seconds - - TestingV3AuthInitialDistDelay 20 seconds - Accelerate initial voting schedule until first consensus is reached. - - - TestingAuthDirTimeToLearnReachability 0 minutes - Consider routers as Running from the start of running an authority. - - - TestingEstimatedDescriptorPropagationTime 0 minutes - Clients try downloading router descriptors from directory caches, - even when they are not 10 minutes old. - - In addition to changing the defaults for these configuration options, - TestingTorNetwork can only be set when a user has manually configured - DirServer lines. - -Test: - - The implementation of this proposal must pass the following tests: - - 1. Set TestingTorNetwork and see if dependent configuration options are - correctly changed. - - tor DataDirectory . ControlPort 9051 TestingTorNetwork 1 DirServer \ - "mydir 127.0.0.1:1234 0000000000000000000000000000000000000000" - telnet 127.0.0.1 9051 - AUTHENTICATE - GETCONF TestingTorNetwork TestingAuthDirTimeToLearnReachability - 250-TestingTorNetwork=1 - 250 TestingAuthDirTimeToLearnReachability=0 - QUIT - - 2. Set TestingTorNetwork and a dependent configuration value to see if - the provided value is used for the dependent option. - - tor DataDirectory . ControlPort 9051 TestingTorNetwork 1 DirServer \ - "mydir 127.0.0.1:1234 0000000000000000000000000000000000000000" \ - TestingAuthDirTimeToLearnReachability 5 - telnet 127.0.0.1 9051 - AUTHENTICATE - GETCONF TestingTorNetwork TestingAuthDirTimeToLearnReachability - 250-TestingTorNetwork=1 - 250 TestingAuthDirTimeToLearnReachability=5 - QUIT - - 3. Start with TestingTorNetwork set and change a dependent configuration - option later on. - - tor DataDirectory . ControlPort 9051 TestingTorNetwork 1 DirServer \ - "mydir 127.0.0.1:1234 0000000000000000000000000000000000000000" - telnet 127.0.0.1 9051 - AUTHENTICATE - SETCONF TestingAuthDirTimeToLearnReachability=5 - GETCONF TestingAuthDirTimeToLearnReachability - 250 TestingAuthDirTimeToLearnReachability=5 - QUIT - - 4. Start with TestingTorNetwork set and a dependent configuration value, - and reset that dependent configuration value. The result should be - the testing-network specific default value. - - tor DataDirectory . ControlPort 9051 TestingTorNetwork 1 DirServer \ - "mydir 127.0.0.1:1234 0000000000000000000000000000000000000000" \ - TestingAuthDirTimeToLearnReachability 5 - telnet 127.0.0.1 9051 - AUTHENTICATE - GETCONF TestingAuthDirTimeToLearnReachability - 250 TestingAuthDirTimeToLearnReachability=5 - RESETCONF TestingAuthDirTimeToLearnReachability - GETCONF TestingAuthDirTimeToLearnReachability - 250 TestingAuthDirTimeToLearnReachability=0 - QUIT - - 5. Leave TestingTorNetwork unset and check if dependent configuration - options are left unchanged. - - tor DataDirectory . ControlPort 9051 DirServer \ - "mydir 127.0.0.1:1234 0000000000000000000000000000000000000000" - telnet 127.0.0.1 9051 - AUTHENTICATE - GETCONF TestingTorNetwork TestingAuthDirTimeToLearnReachability - 250-TestingTorNetwork=0 - 250 TestingAuthDirTimeToLearnReachability=1800 - QUIT - - 6. Leave TestingTorNetwork unset, but set dependent configuration option - which should fail. - - tor DataDirectory . ControlPort 9051 DirServer \ - "mydir 127.0.0.1:1234 0000000000000000000000000000000000000000" \ - TestingAuthDirTimeToLearnReachability 0 - [warn] Failed to parse/validate config: - TestingAuthDirTimeToLearnReachability may only be changed in testing - Tor networks! - - 7. Start with TestingTorNetwork unset and change dependent configuration - option later on which should fail. - - tor DataDirectory . ControlPort 9051 DirServer \ - "mydir 127.0.0.1:1234 0000000000000000000000000000000000000000" - telnet 127.0.0.1 9051 - AUTHENTICATE - SETCONF TestingAuthDirTimeToLearnReachability=0 - 513 Unacceptable option value: TestingAuthDirTimeToLearnReachability - may only be changed in testing Tor networks! - - 8. Start with TestingTorNetwork unset and set it later on which should - fail. - - tor DataDirectory . ControlPort 9051 DirServer \ - "mydir 127.0.0.1:1234 0000000000000000000000000000000000000000" - telnet 127.0.0.1 9051 - AUTHENTICATE - SETCONF TestingTorNetwork=1 - 553 Transition not allowed: While Tor is running, changing - TestingTorNetwork is not allowed. - - 9. Start with TestingTorNetwork set and unset it later on which should - fail. - - tor DataDirectory . ControlPort 9051 TestingTorNetwork 1 DirServer \ - "mydir 127.0.0.1:1234 0000000000000000000000000000000000000000" - telnet 127.0.0.1 9051 - AUTHENTICATE - RESETCONF TestingTorNetwork - 513 Unacceptable option value: TestingV3AuthInitialVotingInterval may - only be changed in testing Tor networks! - - 10. Set TestingTorNetwork, but do not provide an alternate DirServer - which should fail. - - tor DataDirectory . ControlPort 9051 TestingTorNetwork 1 - [warn] Failed to parse/validate config: TestingTorNetwork may only be - configured in combination with a non-default set of DirServers. - diff --git a/doc/spec/proposals/136-legacy-keys.txt b/doc/spec/proposals/136-legacy-keys.txt deleted file mode 100644 index f2b1b5c7f..000000000 --- a/doc/spec/proposals/136-legacy-keys.txt +++ /dev/null @@ -1,100 +0,0 @@ -Filename: 136-legacy-keys.txt -Title: Mass authority migration with legacy keys -Author: Nick Mathewson -Created: 13-May-2008 -Status: Closed -Implemented-In: 0.2.0.x - -Overview: - - This document describes a mechanism to change the keys of more than - half of the directory servers at once without breaking old clients - and caches immediately. - -Motivation: - - If a single authority's identity key is believed to be compromised, - the solution is obvious: remove that authority from the list, - generate a new certificate, and treat the new cert as belonging to a - new authority. This approach works fine so long as less than 1/2 of - the authority identity keys are bad. - - Unfortunately, the mass-compromise case is possible if there is a - sufficiently bad bug in Tor or in any OS used by a majority of v3 - authorities. Let's be prepared for it! - - We could simply stop using the old keys and start using new ones, - and tell all clients running insecure versions to upgrade. - Unfortunately, this breaks our cacheing system pretty badly, since - caches won't cache a consensus that they don't believe in. It would - be nice to have everybody become secure the moment they upgrade to a - version listing the new authority keys, _without_ breaking upgraded - clients until the caches upgrade. - - So, let's come up with a way to provide a time window where the - consensuses are signed with the new keys and with the old. - -Design: - - We allow directory authorities to list a single "legacy key" - fingerprint in their votes. Each authority may add a single legacy - key. The format for this line is: - - legacy-dir-key FINGERPRINT - - We describe a new consensus method for generating directory - consensuses. This method is consensus method "3". - - When the authorities decide to use method "3" (as described in 3.4.1 - of dir-spec.txt), for every included vote with a legacy-dir-key line, - the consensus includes an extra dir-source line. The fingerprint in - this extra line is as in the legacy-dir-key line. The ports and - addresses are in the dir-source line. The nickname is as in the - dir-source line, with the string "-legacy" appended. - - [We need to include this new dir-source line because the code - won't accept or preserve signatures from authorities not listed - as contributing to the consensus.] - - Authorities using legacy dir keys include two signatures on their - consensuses: one generated with a signing key signed with their real - signing key, and another generated with a signing key signed with - another signing key attested to by their identity key. These - signing keys MUST be different. Authorities MUST serve both - certificates if asked. - -Process: - - In the event of a mass key failure, we'll follow the following - (ugly) procedure: - - All affected authorities generate new certificates and identity - keys, and circulate their new dirserver lines. They copy their old - certificates and old broken keys, but put them in new "legacy - key files". - - At the earliest time that can be arranged, the authorities - replace their signing keys, identity keys, and certificates - with the new uncompromised versions, and update to the new list - of dirserer lines. - - They add an "V3DirAdvertiseLegacyKey 1" option to their torrc. - - Now, new consensuses will be generated using the new keys, but - the results will also be signed with the old keys. - - Clients and caches are told they need to upgrade, and given a - time window to do so. - - At the end of the time window, authorities remove the - V3DirAdvertiseLegacyKey option. - -Notes: - - It might be good to get caches to cache consensuses that they do not - believe in. I'm not sure the best way of how to do this. - - It's a superficially neat idea to have new signing keys and have - them signed by the new and by the old authority identity keys. This - breaks some code, though, and doesn't actually gain us anything, - since we'd still need to include each signature twice. - - It's also a superficially neat idea, if identity keys and signing - keys are compromised, to at least replace all the signing keys. - I don't think this achieves us anything either, though. - - diff --git a/doc/spec/proposals/137-bootstrap-phases.txt b/doc/spec/proposals/137-bootstrap-phases.txt deleted file mode 100644 index 18d3dfae1..000000000 --- a/doc/spec/proposals/137-bootstrap-phases.txt +++ /dev/null @@ -1,237 +0,0 @@ -Filename: 137-bootstrap-phases.txt -Title: Keep controllers informed as Tor bootstraps -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 07-Jun-2008 -Status: Closed -Implemented-In: 0.2.1.x - -1. Overview. - - Tor has many steps to bootstrapping directory information and - initial circuits, but from the controller's perspective we just have - a coarse-grained "CIRCUIT_ESTABLISHED" status event. Tor users with - slow connections or with connectivity problems can wait a long time - staring at the yellow onion, wondering if it will ever change color. - - This proposal describes a new client status event so Tor can give - more details to the controller. Section 2 describes the changes to the - controller protocol; Section 3 describes Tor's internal bootstrapping - phases when everything is going correctly; Section 4 describes when - Tor detects a problem and issues a bootstrap warning; Section 5 covers - suggestions for how controllers should display the results. - -2. Controller event syntax. - - The generic status event is: - - "650" SP StatusType SP StatusSeverity SP StatusAction - [SP StatusArguments] CRLF - - So in this case we send - 650 STATUS_CLIENT NOTICE/WARN BOOTSTRAP \ - PROGRESS=num TAG=Keyword SUMMARY=String \ - [WARNING=String REASON=Keyword COUNT=num RECOMMENDATION=Keyword] - - The arguments MAY appear in any order. Controllers MUST accept unrecognized - arguments. - - "Progress" gives a number between 0 and 100 for how far through - the bootstrapping process we are. "Summary" is a string that can be - displayed to the user to describe the *next* task that Tor will tackle, - i.e., the task it is working on after sending the status event. "Tag" - is an optional string that controllers can use to recognize bootstrap - phases from Section 3, if they want to do something smarter than just - blindly displaying the summary string. - - The severity describes whether this is a normal bootstrap phase - (severity notice) or an indication of a bootstrapping problem - (severity warn). If severity warn, it should also include a "warning" - argument string with any hints Tor has to offer about why it's having - troubles bootstrapping, a "reason" string that lists one of the reasons - allowed in the ORConn event, a "count" number that tells how many - bootstrap problems there have been so far at this phase, and a - "recommendation" keyword to indicate how the controller ought to react. - -3. The bootstrap phases. - - This section describes the various phases currently reported by - Tor. Controllers should not assume that the percentages and tags listed - here will continue to match up, or even that the tags will stay in - the same order. Some phases might also be skipped (not reported) if the - associated bootstrap step is already complete, or if the phase no longer - is necessary. Only "starting" and "done" are guaranteed to exist in all - future versions. - - Current Tor versions enter these phases in order, monotonically; - future Tors MAY revisit earlier stages. - - Phase 0: - tag=starting summary="starting" - - Tor starts out in this phase. - - Phase 5: - tag=conn_dir summary="Connecting to directory mirror" - - Tor sends this event as soon as Tor has chosen a directory mirror --- - one of the authorities if bootstrapping for the first time or after - a long downtime, or one of the relays listed in its cached directory - information otherwise. - - Tor will stay at this phase until it has successfully established - a TCP connection with some directory mirror. Problems in this phase - generally happen because Tor doesn't have a network connection, or - because the local firewall is dropping SYN packets. - - Phase 10 - tag=handshake_dir summary="Finishing handshake with directory mirror" - - This event occurs when Tor establishes a TCP connection with a relay used - as a directory mirror (or its https proxy if it's using one). Tor remains - in this phase until the TLS handshake with the relay is finished. - - Problems in this phase generally happen because Tor's firewall is - doing more sophisticated MITM attacks on it, or doing packet-level - keyword recognition of Tor's handshake. - - Phase 15: - tag=onehop_create summary="Establishing one-hop circuit for dir info" - - Once TLS is finished with a relay, Tor will send a CREATE_FAST cell - to establish a one-hop circuit for retrieving directory information. - It will remain in this phase until it receives the CREATED_FAST cell - back, indicating that the circuit is ready. - - Phase 20: - tag=requesting_status summary="Asking for networkstatus consensus" - - Once we've finished our one-hop circuit, we will start a new stream - for fetching the networkstatus consensus. We'll stay in this phase - until we get the 'connected' relay cell back, indicating that we've - established a directory connection. - - Phase 25: - tag=loading_status summary="Loading networkstatus consensus" - - Once we've established a directory connection, we will start fetching - the networkstatus consensus document. This could take a while; this - phase is a good opportunity for using the "progress" keyword to indicate - partial progress. - - This phase could stall if the directory mirror we picked doesn't - have a copy of the networkstatus consensus so we have to ask another, - or it does give us a copy but we don't find it valid. - - Phase 40: - tag=loading_keys summary="Loading authority key certs" - - Sometimes when we've finished loading the networkstatus consensus, - we find that we don't have all the authority key certificates for the - keys that signed the consensus. At that point we put the consensus we - fetched on hold and fetch the keys so we can verify the signatures. - - Phase 45 - tag=requesting_descriptors summary="Asking for relay descriptors" - - Once we have a valid networkstatus consensus and we've checked all - its signatures, we start asking for relay descriptors. We stay in this - phase until we have received a 'connected' relay cell in response to - a request for descriptors. - - Phase 50: - tag=loading_descriptors summary="Loading relay descriptors" - - We will ask for relay descriptors from several different locations, - so this step will probably make up the bulk of the bootstrapping, - especially for users with slow connections. We stay in this phase until - we have descriptors for at least 1/4 of the usable relays listed in - the networkstatus consensus. This phase is also a good opportunity to - use the "progress" keyword to indicate partial steps. - - Phase 80: - tag=conn_or summary="Connecting to entry guard" - - Once we have a valid consensus and enough relay descriptors, we choose - some entry guards and start trying to build some circuits. This step - is similar to the "conn_dir" phase above; the only difference is - the context. - - If a Tor starts with enough recent cached directory information, - its first bootstrap status event will be for the conn_or phase. - - Phase 85: - tag=handshake_or summary="Finishing handshake with entry guard" - - This phase is similar to the "handshake_dir" phase, but it gets reached - if we finish a TCP connection to a Tor relay and we have already reached - the "conn_or" phase. We'll stay in this phase until we complete a TLS - handshake with a Tor relay. - - Phase 90: - tag=circuit_create "Establishing circuits" - - Once we've finished our TLS handshake with an entry guard, we will - set about trying to make some 3-hop circuits in case we need them soon. - - Phase 100: - tag=done summary="Done" - - A full 3-hop circuit has been established. Tor is ready to handle - application connections now. - -4. Bootstrap problem events. - - When an OR Conn fails, we send a "bootstrap problem" status event, which - is like the standard bootstrap status event except with severity warn. - We include the same progress, tag, and summary values as we would for - a normal bootstrap event, but we also include "warning", "reason", - "count", and "recommendation" key/value combos. - - The "reason" values are long-term-stable controller-facing tags to - identify particular issues in a bootstrapping step. The warning - strings, on the other hand, are human-readable. Controllers SHOULD - NOT rely on the format of any warning string. Currently the possible - values for "recommendation" are either "ignore" or "warn" -- if ignore, - the controller can accumulate the string in a pile of problems to show - the user if the user asks; if warn, the controller should alert the - user that Tor is pretty sure there's a bootstrapping problem. - - Currently Tor uses recommendation=ignore for the first nine bootstrap - problem reports for a given phase, and then uses recommendation=warn - for subsequent problems at that phase. Hopefully this is a good - balance between tolerating occasional errors and reporting serious - problems quickly. - -5. Suggested controller behavior. - - Controllers should start out with a yellow onion or the equivalent - ("starting"), and then watch for either a bootstrap status event - (meaning the Tor they're using is sufficiently new to produce them, - and they should load up the progress bar or whatever they plan to use - to indicate progress) or a circuit_established status event (meaning - bootstrapping is finished). - - In addition to a progress bar in the display, controllers should also - have some way to indicate progress even when no controller window is - open. For example, folks using Tor Browser Bundle in hostile Internet - cafes don't want a big splashy screen up. One way to let the user keep - informed of progress in a more subtle way is to change the task tray - icon and/or tooltip string as more bootstrap events come in. - - Controllers should also have some mechanism to alert their user when - bootstrapping problems are reported. Perhaps we should gather a set of - help texts and the controller can send the user to the right anchor in a - "bootstrapping problems" page in the controller's help subsystem? - -6. Getting up to speed when the controller connects. - - There's a new "GETINFO /status/bootstrap-phase" option, which returns - the most recent bootstrap phase status event sent. Specifically, - it returns a string starting with either "NOTICE BOOTSTRAP ..." or - "WARN BOOTSTRAP ...". - - Controllers should use this getinfo when they connect or attach to - Tor to learn its current state. - diff --git a/doc/spec/proposals/138-remove-down-routers-from-consensus.txt b/doc/spec/proposals/138-remove-down-routers-from-consensus.txt deleted file mode 100644 index a07764d53..000000000 --- a/doc/spec/proposals/138-remove-down-routers-from-consensus.txt +++ /dev/null @@ -1,51 +0,0 @@ -Filename: 138-remove-down-routers-from-consensus.txt -Title: Remove routers that are not Running from consensus documents -Version: $Revision$ -Last-Modified: $Date$ -Author: Peter Palfrader -Created: 11-Jun-2008 -Status: Closed -Implemented-In: 0.2.1.2-alpha - -1. Overview. - - Tor directory authorities hourly vote and agree on a consensus document - which lists all the routers on the network together with some of their - basic properties, like if a router is an exit node, whether it is - stable or whether it is a version 2 directory mirror. - - One of the properties given with each router is the 'Running' flag. - Clients do not use routers that are not listed as running. - - This proposal suggests that routers without the Running flag are not - listed at all. - -2. Current status - - At a typical bootstrap a client downloads a 140KB consensus, about - 10KB of certificates to verify that consensus, and about 1.6MB of - server descriptors, about 1/4 of which it requires before it will - start building circuits. - - Another proposal deals with how to get that huge 1.6MB fraction to - effectively zero (by downloading only individual descriptors, on - demand). Should that get successfully implemented that will leave the - 140KB compressed consensus as a large fraction of what a client needs - to get in order to work. - - About one third of the routers listed in a consensus are not running - and will therefore never be used by clients who use this consensus. - Not listing those routers will save about 30% to 40% in size. - -3. Proposed change - - Authority directory servers produce vote documents that include all - the servers they know about, running or not, like they currently - do. In addition these vote documents also state that the authority - supports a new consensus forming method (method number 4). - - If more than two thirds of votes that an authority has received claim - they support method 4 then this new method will be used: The - consensus document is formed like before but a new last step removes - all routers from the listing that are not marked as Running. - diff --git a/doc/spec/proposals/139-conditional-consensus-download.txt b/doc/spec/proposals/139-conditional-consensus-download.txt deleted file mode 100644 index 941f5ad6b..000000000 --- a/doc/spec/proposals/139-conditional-consensus-download.txt +++ /dev/null @@ -1,94 +0,0 @@ -Filename: 139-conditional-consensus-download.txt -Title: Download consensus documents only when it will be trusted -Author: Peter Palfrader -Created: 2008-04-13 -Status: Closed -Implemented-In: 0.2.1.x - -Overview: - - Servers only provide consensus documents to clients when it is known that - the client will trust it. - -Motivation: - - When clients[1] want a new network status consensus they request it - from a Tor server using the URL path /tor/status-vote/current/consensus. - Then after downloading the client checks if this consensus can be - trusted. Whether the client trusts the consensus depends on the - authorities that the client trusts and how many of those - authorities signed the consensus document. - - If the client cannot trust the consensus document it is disregarded - and a new download is tried at a later time. Several hundred - kilobytes of server bandwidth were wasted by this single client's - request. - - With hundreds of thousands of clients this will have undesirable - consequences when the list of authorities has changed so much that a - large number of established clients no longer can trust any consensus - document formed. - -Objective: - - The objective of this proposal is to make clients not download - consensuses they will not trust. - -Proposal: - - The list of authorities that are trusted by a client are encoded in - the URL they send to the directory server when requesting a consensus - document. - - The directory server then only sends back the consensus when more than - half of the authorities listed in the request have signed the - consensus. If it is known that the consensus will not be trusted - a 404 error code is sent back to the client. - - This proposal does not require directory caches to keep more than one - consensus document. This proposal also does not require authorities - to verify the signature on the consensus document of authorities they - do not recognize. - - The new URL scheme to download a consensus is - /tor/status-vote/current/consensus/ where F is a list of - fingerprints, sorted in ascending order, and concatenated using a + - sign. - - Fingerprints are uppercase hexadecimal encodings of the authority - identity key's digest. Servers should also accept requests that - use lower case or mixed case hexadecimal encodings. - - A .z URL for compressed versions of the consensus will be provided - similarly to existing resources and is the URL that usually should - be used by clients. - -Migration: - - The old location of the consensus should continue to work - indefinitely. Not only is it used by old clients, but it is a useful - resource for automated tools that do not particularly care which - authorities have signed the consensus. - - Authorities that are known to the client a priori by being shipped - with the Tor code are assumed to handle this format. - - When downloading a consensus document from caches that do not support this - new format they fall back to the old download location. - - Caches support the new format starting with Tor version 0.2.1.1-alpha. - -Anonymity Implications: - - By supplying the list of authorities a client trusts to the directory - server we leak information (like likely version of Tor client) to the - directory server. In the current system we also leak that we are - very old - by re-downloading the consensus over and over again, but - only when we are so old that we no longer can trust the consensus. - - - -Footnotes: - 1. For the purpose of this proposal a client can be any Tor instance - that downloads a consensus document. This includes relays, - directory caches as well as end users. diff --git a/doc/spec/proposals/140-consensus-diffs.txt b/doc/spec/proposals/140-consensus-diffs.txt deleted file mode 100644 index da63bfe23..000000000 --- a/doc/spec/proposals/140-consensus-diffs.txt +++ /dev/null @@ -1,149 +0,0 @@ -Filename: 140-consensus-diffs.txt -Title: Provide diffs between consensuses -Version: $Revision$ -Last-Modified: $Date$ -Author: Peter Palfrader -Created: 13-Jun-2008 -Status: Accepted -Target: 0.2.2.x - -1. Overview. - - Tor clients and servers need a list of which relays are on the - network. This list, the consensus, is created by authorities - hourly and clients fetch a copy of it, with some delay, hourly. - - This proposal suggests that clients download diffs of consensuses - once they have a consensus instead of hourly downloading a full - consensus. - -2. Numbers - - After implementing proposal 138 which removes nodes that are not - running from the list a consensus document is about 92 kilobytes - in size after compression. - - The diff between two consecutive consensus, in ed format, is on - average 13 kilobytes compressed. - -3. Proposal - -3.1 Clients - - If a client has a consensus that is recent enough it SHOULD - try to download a diff to get the latest consensus rather than - fetching a full one. - - [XXX: what is recent enough? - time delta in hours / size of compressed diff - 0 20 - 1 9650 - 2 17011 - 3 23150 - 4 29813 - 5 36079 - 6 39455 - 7 43903 - 8 48907 - 9 54549 - 10 60057 - 11 67810 - 12 71171 - 13 73863 - 14 76048 - 15 80031 - 16 84686 - 17 89862 - 18 94760 - 19 94868 - 20 94223 - 21 93921 - 22 92144 - 23 90228 - [ size of gzip compressed "diff -e" between the consensus on - 2008-06-01-00:00:00 and the following consensuses that day. - Consensuses have been modified to exclude down routers per - proposal 138. ] - - Data suggests that for the first few hours diffs are very useful, - saving about 60% for the first three hours, 30% for the first 10, - and almost nothing once we are past 16 hours. - ] - -3.2 Servers - - Directory authorities and servers need to keep up to X [XXX: depends - on how long clients try to download diffs per above] old consensus - documents so they can build diffs. They should offer a diff to the - most recent consensus at the URL - - http://tor.noreply.org/tor/status-vote/current/consensus/diff// - - where hash is the full digest of the consensus the client currently - has, and FPRLIST is a list of (abbreviated) fingerprints of - authorities the client trusts. - - Servers will only return a consensus if more than half of the requested - authorities have signed the document, otherwise a 404 error will be sent - back. The fingerprints can be shortened to a length of any multiple of - two, using only the leftmost part of the encoded fingerprint. Tor uses - 3 bytes (6 hex characters) of the fingerprint. (This is just like the - conditional consensus downloads that Tor supports starting with - 0.1.2.1-alpha.) - - If a server cannot offer a diff from the consensus identified by the - hash but has a current consensus it MUST return the full consensus. - - [XXX: what should we do when the client already has the latest - consensus? I can think of the following options: - - send back 3xx not modified - - send back 200 ok and an empty diff - - send back 404 nothing newer here. - - I currently lean towards the empty diff.] - -4. Diff Format - - Diffs start with the token "network-status-diff-version" followed by a - space and the version number, currently "1". - - If a document does not start with network-status-diff it is assumed - to be a full consensus download and would therefore currently start - with "network-status-version 3". - - Following the network-status-diff header line is a diff, or patch, in - limited ed format. We choose this format because it is easy to create - and process with standard tools (patch, diff -e, ed). This will help - us in developing and testing this proposal and it should make future - debugging easier. - - [ If at one point in the future we decide that the space benefits from - a custom diff format outweighs these benefits we can always - introduce a new diff format and offer it at for instance - ../diff2/... ] - - We support the following ed commands, each on a line by itself: - - "d" Delete line n1 - - ",d" Delete lines n1 through n2, including - - "c" Replace line n1 with the following block - - ",c" Replace lines n1 through n2, including, with the - following block. - - "a" Append the following block after line n1. - - "a" Append the following block after the current line. - - "s/.//" Remove the first character in the current line. - - Note that line numbers always apply to the file after all previous - commands have already been applied. - - The "current line" is either the first line of the file, if this is - the first command, the last line of a block we added in an append or - change command, or the line immediate following a set of lines we just - deleted (or the last line of the file if there are no lines after - that). - - The replace and append command take blocks. These blocks are simply - appended to the diff after the line with the command. A line with - just a period (".") ends the block (and is not part of the lines - to add). Note that it is impossible to insert a line with just - a single dot. Recommended procedure is to insert a line with - two dots, then remove the first character of that line using s/.//. diff --git a/doc/spec/proposals/141-jit-sd-downloads.txt b/doc/spec/proposals/141-jit-sd-downloads.txt deleted file mode 100644 index b0c2b2cbc..000000000 --- a/doc/spec/proposals/141-jit-sd-downloads.txt +++ /dev/null @@ -1,325 +0,0 @@ -Filename: 141-jit-sd-downloads.txt -Title: Download server descriptors on demand -Version: $Revision$ -Last-Modified: $Date$ -Author: Peter Palfrader -Created: 15-Jun-2008 -Status: Draft - -1. Overview - - Downloading all server descriptors is the most expensive part - of bootstrapping a Tor client. These server descriptors currently - amount to about 1.5 Megabytes of data, and this size will grow - linearly with network size. - - Fetching all these server descriptors takes a long while for people - behind slow network connections. It is also a considerable load on - our network of directory mirrors. - - This document describes proposed changes to the Tor network and - directory protocol so that clients will no longer need to download - all server descriptors. - - These changes consist of moving load balancing information into - network status documents, implementing a means to download server - descriptors on demand in an anonymity-preserving way, and dealing - with exit node selection. - -2. What is in a server descriptor - - When a Tor client starts the first thing it will try to get is a - current network status document: a consensus signed by a majority - of directory authorities. This document is currently about 100 - Kilobytes in size, tho it will grow linearly with network size. - This document lists all servers currently running on the network. - The Tor client will then try to get a server descriptor for each - of the running servers. All server descriptors currently amount - to about 1.5 Megabytes of downloads. - - A Tor client learns several things about a server from its descriptor. - Some of these it already learned from the network status document - published by the authorities, but the server descriptor contains it - again in a single statement signed by the server itself, not just by - the directory authorities. - - Tor clients use the information from server descriptors for - different purposes, which are considered in the following sections. - - #three ways: One, to determine if a server will be able to handle - #this client's request; two, to actually communicate or use the server; - #three, for load balancing decisions. - # - #These three points are considered in the following subsections. - -2.1 Load balancing - - The Tor load balancing mechanism is quite complex in its details, but - it has a simple goal: The more traffic a server can handle the more - traffic it should get. That means the more traffic a server can - handle the more likely a client will use it. - - For this purpose each server descriptor has bandwidth information - which tries to convey a server's capacity to clients. - - Currently we weigh servers differently for different purposes. There - is a weigh for when we use a server as a guard node (our entry to the - Tor network), there is one weigh we assign servers for exit duties, - and a third for when we need intermediate (middle) nodes. - -2.2 Exit information - - When a Tor wants to exit to some resource on the internet it will - build a circuit to an exit node that allows access to that resource's - IP address and TCP Port. - - When building that circuit the client can make sure that the circuit - ends at a server that will be able to fulfill the request because the - client already learned of all the servers' exit policies from their - descriptors. - -2.3 Capability information - - Server descriptors contain information about the specific version or - the Tor protocol they understand [proposal 105]. - - Furthermore the server descriptor also contains the exact version of - the Tor software that the server is running and some decisions are - made based on the server version number (for instance a Tor client - will only make conditional consensus requests [proposal 139] when - talking to Tor servers version 0.2.1.1-alpha or later). - -2.4 Contact/key information - - A server descriptor lists a server's IP address and TCP ports on which - it accepts onion and directory connections. Furthermore it contains - the onion key (a short lived RSA key to which clients encrypt CREATE - cells). - -2.5 Identity information - - A Tor client learns the digest of a server's key from the network - status document. Once it has a server descriptor this descriptor - contains the full RSA identity key of the server. Clients verify - that 1) the digest of the identity key matches the expected digest - it got from the consensus, and 2) that the signature on the descriptor - from that key is valid. - - -3. No longer require clients to have copies of all SDs - -3.1 Load balancing info in consensus documents - - One of the reasons why clients download all server descriptors is for - doing load proper load balancing as described in 2.1. In order for - clients to not require all server descriptors this information will - have to move into the network status document. - - Consensus documents will have a new line per router similar - to the "r", "s", and "v" lines that already exist. This line - will convey weight information to clients. - - "w Bandwidth=193" - - The bandwidth number is the lesser of observed bandwidth and bandwidth - rate limit from the server descriptor that the "r" line referenced by - digest (1st and 3rd field of the bandwidth line in the descriptor). - It is given in kilobytes per second so the byte value in the - descriptor has to be divided by 1024 (and is then truncated, i.e. - rounded down). - - Authorities will cap the bandwidth number at some arbitrary value, - currently 10MB/sec. If a router claims a larger bandwidth an - authority's vote will still only show Bandwidth=10240. - - The consensus value for bandwidth is the median of all bandwidth - numbers given in votes. In case of an even number of votes we use - the lower median. (Using this procedure allows us to change the - cap value more easily.) - - Clients should believe the bandwidth as presented in the consensus, - not capping it again. - -3.2 Fetching descriptors on demand - - As described in 2.4 a descriptor lists IP address, OR- and Dir-Port, - and the onion key for a server. - - A client already knows the IP address and the ports from the consensus - documents, but without the onion key it will not be able to send - CREATE/EXTEND cells for that server. Since the client needs the onion - key it needs the descriptor. - - If a client only downloaded a few descriptors in an observable manner - then that would leak which nodes it was going to use. - - This proposal suggests the following: - - 1) when connecting to a guard node for which the client does not - yet have a cached descriptor it requests the descriptor it - expects by hash. (The consensus document that the client holds - has a hash for the descriptor of this server. We want exactly - that descriptor, not a different one.) - - It does that by sending a RELAY_REQUEST_SD cell. - - A client MAY cache the descriptor of the guard node so that it does - not need to request it every single time it contacts the guard. - - 2) when a client wants to extend a circuit that currently ends in - server B to a new next server C, the client will send a - RELAY_REQUEST_SD cell to server B. This cell contains in its - payload the hash of a server descriptor the client would like - to obtain (C's server descriptor). The server sends back the - descriptor and the client can now form a valid EXTEND/CREATE cell - encrypted to C's onion key. - - Clients MUST NOT cache such descriptors. If they did they might - leak that they already extended to that server at least once - before. - - Replies to RELAY_REQUEST_SD requests need to be padded to some - constant upper limit in order to conceal a client's destination - from anybody who might be counting cells/bytes. - - RELAY_REQUEST_SD cells contain the following information: - - hash of the server descriptor requested - - hash of the identity digest of the server for which we want the SD - - IP address and OR-port or the server for which we want the SD - - padding factor - the number of cells we want the answer - padded to. - [XXX this just occured to me and it might be smart. or it might - be stupid. clients would learn the padding factor they want - to use from the consensus document. This allows us to grow - the replies later on should SDs become larger.] - [XXX: figure out a decent padding size] - -3.3 Protocol versions - - Server descriptors contain optional information of supported - link-level and circuit-level protocols in the form of - "opt protocols Link 1 2 Circuit 1". These are not currently needed - and will probably eventually move into the "v" (version) line in - the consensus. This proposal does not deal with them. - - Similarly a server descriptor contains the version number of - a Tor node. This information is already present in the consensus - and is thus available to all clients immediately. - -3.4 Exit selection - - Currently finding an appropriate exit node for a user's request is - easy for a client because it has complete knowledge of all the exit - policies of all servers on the network. - - The consensus document will once again be extended to contain the - information required by clients. This information will be a summary - of each node's exit policy. The exit policy summary will only contain - the list of ports to which a node exits to most destination IP - addresses. - - A summary should claim a router exits to a specific TCP port if, - ignoring private IP addresses, the exit policy indicates that the - router would exit to this port to most IP address. either two /8 - netblocks, or one /8 and a couple of /12s or any other combination). - The exact algorith used is this: Going through all exit policy items - - ignore any accept that is not for all IP addresses ("*"), - - ignore rejects for these netblocks (exactly, no subnetting): - 0.0.0.0/8, 169.254.0.0/16, 127.0.0.0/8, 192.168.0.0/16, 10.0.0.0/8, - and 172.16.0.0/12m - - for each reject count the number of IP addresses rejected against - the affected ports, - - once we hit an accept for all IP addresses ("*") add the ports in - that policy item to the list of accepted ports, if they don't have - more than 2^25 IP addresses (that's two /8 networks) counted - against them (i.e. if the router exits to a port to everywhere but - at most two /8 networks). - - An exit policy summary will be included in votes and consensus as a - new line attached to each exit node. The line will have the format - "p" "accept"|"reject" - where portlist is a comma seperated list of single port numbers or - portranges (e.g. "22,80-88,1024-6000,6667"). - - Whether the summary shows the list of accepted ports or the list of - rejected ports depends on which list is shorter (has a shorter string - representation). In case of ties we choose the list of accepted - ports. As an exception to this rule an allow-all policy is - represented as "accept 1-65535" instead of "reject " and a reject-all - policy is similarly given as "reject 1-65535". - - Summary items are compressed, that is instead of "80-88,89-100" there - only is a single item of "80-100", similarly instead of "20,21" a - summary will say "20-21". - - Port lists are sorted in ascending order. - - The maximum allowed length of a policy summary (including the "accept " - or "reject ") is 1000 characters. If a summary exceeds that length we - use an accept-style summary and list as much of the port list as is - possible within these 1000 bytes. - -3.4.1 Consensus selection - - When building a consensus, authorities have to agree on a digest of - the server descriptor to list in the router line for each router. - This is documented in dir-spec section 3.4. - - All authorities that listed that agreed upon descriptor digest in - their vote should also list the same exit policy summary - or list - none at all if the authority has not been upgraded to list that - information in their vote. - - If we have votes with matching server descriptor digest of which at - least one of them has an exit policy then we differ between two cases: - a) all authorities agree (or abstained) on the policy summary, and we - use the exit policy summary that they all listed in their vote, - b) something went wrong (or some authority is playing foul) and we - have different policy summaries. In that case we pick the one - that is most commonly listed in votes with the matching - descriptor. We break ties in favour of the lexigraphically larger - vote. - - If none one of the votes with a matching server descriptor digest has - an exit policy summary we use the most commonly listed one in all - votes, breaking ties like in case b above. - -3.4.2 Client behaviour - - When choosing an exit node for a specific request a Tor client will - choose from the list of nodes that exit to the requested port as given - by the consensus document. If a client has additional knowledge (like - cached full descriptors) that indicates the so chosen exit node will - reject the request then it MAY use that knowledge (or not include such - nodes in the selection to begin with). However, clients MUST NOT use - nodes that do not list the port as accepted in the summary (but for - which they know that the node would exit to that address from other - sources, like a cached descriptor). - - An exception to this is exit enclave behaviour: A client MAY use the - node at a specific IP address to exit to any port on the same address - even if that node is not listed as exiting to the port in the summary. - -4. Migration - -4.1 Consensus document changes. - - The consensus will need to include - - bandwidth information (see 3.1) - - exit policy summaries (3.4) - - A new consensus method (number TBD) will be chosen for this. - -5. Future possibilities - - This proposal still requires that all servers have the descriptors of - every other node in the network in order to answer RELAY_REQUEST_SD - cells. These cells are sent when a circuit is extended from ending at - node B to a new node C. In that case B would have to answer a - RELAY_REQUEST_SD cell that asks for C's server descriptor (by SD digest). - - In order to answer that request B obviously needs a copy of C's server - descriptor. The RELAY_REQUEST_SD cell already has all the info that - B needs to contact C so it can ask about the descriptor before passing it - back to the client. - diff --git a/doc/spec/proposals/142-combine-intro-and-rend-points.txt b/doc/spec/proposals/142-combine-intro-and-rend-points.txt deleted file mode 100644 index 3456b285a..000000000 --- a/doc/spec/proposals/142-combine-intro-and-rend-points.txt +++ /dev/null @@ -1,279 +0,0 @@ -Filename: 142-combine-intro-and-rend-points.txt -Title: Combine Introduction and Rendezvous Points -Version: $Revision$ -Last-Modified: $Date$ -Author: Karsten Loesing, Christian Wilms -Created: 27-Jun-2008 -Status: Dead - -Change history: - - 27-Jun-2008 Initial proposal for or-dev - 04-Jul-2008 Give first security property the new name "Responsibility" - and change new cell formats according to rendezvous protocol - version 3 draft. - 19-Jul-2008 Added comment by Nick (but no solution, yet) that sharing of - circuits between multiple clients is not supported by Tor. - -Overview: - - Establishing a connection to a hidden service currently involves two Tor - relays, introduction and rendezvous point, and 10 more relays distributed - over four circuits to connect to them. The introduction point is - established in the mid-term by a hidden service to transfer introduction - requests from client to the hidden service. The rendezvous point is set - up by the client for a single hidden service request and actually - transfers end-to-end encrypted application data between client and hidden - service. - - There are some reasons for separating the two roles of introduction and - rendezvous point: (1) Responsibility: A relay shall not be made - responsible that it relays data for a certain hidden service; in the - original design as described in [1] an introduction point relays no - application data, and a rendezvous points neither knows the hidden - service nor can it decrypt the data. (2) Scalability: The hidden service - shall not have to maintain a number of open circuits proportional to the - expected number of client requests. (3) Attack resistance: The effect of - an attack on the only visible parts of a hidden service, its introduction - points, shall be as small as possible. - - However, elimination of a separate rendezvous connection as proposed by - Øverlier and Syverson [2] is the most promising approach to improve the - delay in connection establishment. From all substeps of connection - establishment extending a circuit by only a single hop is responsible for - a major part of delay. Reducing on-demand circuit extensions from two to - one results in a decrease of mean connection establishment times from 39 - to 29 seconds [3]. Particularly, eliminating the delay on hidden-service - side allows the client to better observe progress of connection - establishment, thus allowing it to use smaller timeouts. Proposal 114 - introduced new introduction keys for introduction points and provides for - user authorization data in hidden service descriptors; it will be shown - in this proposal that introduction keys in combination with new - introduction cookies provide for the first security property - responsibility. Further, eliminating the need for a separate introduction - connection benefits the overall network load by decreasing the number of - circuit extensions. After all, having only one connection between client - and hidden service reduces the overall protocol complexity. - -Design: - - 1. Hidden Service Configuration - - Hidden services should be able to choose whether they would like to use - this protocol. This might be opt-in for 0.2.1.x and opt-out for later - major releases. - - 2. Contact Point Establishment - - When preparing a hidden service, a Tor client selects a set of relays to - act as contact points instead of introduction points. The contact point - combines both roles of introduction and rendezvous point as proposed in - [2]. The only requirement for a relay to be picked as contact point is - its capability of performing this role. This can be determined from the - Tor version number that needs to be equal or higher than the first - version that implements this proposal. - - The easiest way to implement establishment of contact points is to - introduce v2 ESTABLISH_INTRO cells. By convention, the relay recognizes - version 2 ESTABLISH_INTRO cells as requests to establish a contact point - rather than an introduction point. - - V Format byte: set to 255 [1 octet] - V Version byte: set to 2 [1 octet] - KLEN Key length [2 octets] - PK Public introduction key [KLEN octets] - HS Hash of session info [20 octets] - SIG Signature of above information [variable] - - The hidden service does not create a fixed number of contact points, like - 3 in the current protocol. It uses a minimum of 3 contact points, but - increases this number depending on the history of client requests within - the last hour. The hidden service also increases this number depending on - the frequency of failing contact points in order to defend against - attacks on its contact points. When client authorization as described in - proposal 121 is used, a hidden service can also use the number of - authorized clients as first estimate for the required number of contact - points. - - 3. Hidden Service Descriptor Creation - - A hidden service needs to issue a fresh introduction cookie for each - established introduction point. By requiring clients to use this cookie - in a later connection establishment, an introduction point cannot access - the hidden service that it works for. Together with the fresh - introduction key that was introduced in proposal 114, this reduces - responsibility of a contact point for a specific hidden service. - - The v2 hidden service descriptor format contains an - "intro-authentication" field that may contain introduction-point specific - keys. The hidden service creates a random string, comparable to the - rendezvous cookie, and includes it in the descriptor as introduction - cookie for auth-type "1". By convention, clients recognize existence of - auth-type 1 as possibility to connect to a hidden service via a contact - point rather than an introduction point. Older clients that do not - understand this new protocol simply ignore that cookie. - - 4. Connection Establishment - - When establishing a connection to a hidden service a client learns about - the capability of using the new protocol from the hidden service - descriptor. It may choose whether to use this new protocol or not, - whereas older clients cannot understand the new capability and can only - use the current protocol. Client using version 0.2.1.x should be able to - opt-in for using the new protocol, which should change to opt-out for - later major releases. - - When using the new capability the client creates a v2 INTRODUCE1 cell - that extends an unversioned INTRODUCE1 cell by adding the content of an - ESTABLISH_RENDEZVOUS cell. Further, the client sends this cell using the - new cell type 41 RELAY_INTRODUCE1_VERSIONED to the introduction point, - because unversioned and versioned INTRODUCE1 cells are indistinguishable: - - Cleartext - V Version byte: set to 2 [1 octet] - PK_ID Identifier for Bob's PK [20 octets] - RC Rendezvous cookie [20 octets] - Encrypted to introduction key: - VER Version byte: set to 3. [1 octet] - AUTHT The auth type that is supported [1 octet] - AUTHL Length of auth data [2 octets] - AUTHD Auth data [variable] - RC Rendezvous cookie [20 octets] - g^x Diffie-Hellman data, part 1 [128 octets] - - The cleartext part contains the rendezvous cookie that the contact point - remembers just as a rendezvous point would do. - - The encrypted part contains the introduction cookie as auth data for the - auth type 1. The rendezvous cookie is contained as before, but there is - no further rendezvous point information, as there is no separate - rendezvous point. - - 5. Rendezvous Establishment - - The contact point recognizes a v2 INTRODUCE1 cell with auth type 1 as a - request to be used in the new protocol. It remembers the contained - rendezvous cookie, replies to the client with an INTRODUCE_ACK cell - (omitting the RENDEZVOUS_ESTABLISHED cell), and forwards the encrypted - part of the INTRODUCE1 cell as INTRODUCE2 cell to the hidden service. - - 6. Introduction at Hidden Service - - The hidden services recognizes an INTRODUCE2 cell containing an - introduction cookie as authorization data. In this case, it does not - extend a circuit to a rendezvous point, but sends a RENDEZVOUS1 cell - directly back to its contact point as usual. - - 7. Rendezvous at Contact Point - - The contact point processes a RENDEZVOUS1 cell just as a rendezvous point - does. The only difference is that the hidden-service-side circuit is not - exclusive for the client connection, but shared among multiple client - connections. - - [Tor does not allow sharing of a single circuit among multiple client - connections easily. We need to think about a smart and efficient way to - implement this. Comment by Nick. -KL] - -Security Implications: - - (1) Responsibility - - One of the original reasons for the separation of introduction and - rendezvous points is that a relay shall not be made responsible that it - relays data for a certain hidden service. In the current design an - introduction point relays no application data and a rendezvous points - neither knows the hidden service nor can it decrypt the data. - - This property is also fulfilled in this new design. A contact point only - learns a fresh introduction key instead of the hidden service key, so - that it cannot recognize a hidden service. Further, the introduction - cookie, which is unknown to the contact point, prevents it from accessing - the hidden service itself. The only way for a contact point to access a - hidden service is to look up whether it is contained in the descriptors - of known hidden services. A contact point cannot directly be made - responsible for which hidden service it is working. In addition to that, - it cannot learn the data that it transfers, because all communication - between client and hidden service are end-to-end encrypted. - - (2) Scalability - - Another goal of the existing hidden service protocol is that a hidden - service does not have to maintain a number of open circuits proportional - to the expected number of client requests. The rationale behind this is - better scalability. - - The new protocol eliminates the need for a hidden service to extend - circuits on demand, which has a positive effect on circuits establishment - times and overall network load. The solution presented here to establish - a number of contact points proportional to the history of connection - requests reduces the number of circuits to a minimum number that fits the - hidden service's needs. - - (3) Attack resistance - - The third goal of separating introduction and rendezvous points is to - limit the effect of an attack on the only visible parts of a hidden - service which are the contact points in this protocol. - - In theory, the new protocol is more vulnerable to this attack. An - attacker who can take down a contact point does not only eliminate an - access point to the hidden service, but also breaks current client - connections to the hidden service using that contact point. - - Øverlier and Syverson proposed the concept of valet nodes as additional - safeguard for introduction/contact points [4]. Unfortunately, this - increases hidden service protocol complexity conceptually and from an - implementation point of view. Therefore, it is not included in this - proposal. - - However, in practice attacking a contact point (or introduction point) is - not as rewarding as it might appear. The cost for a hidden service to set - up a new contact point and publish a new hidden service descriptor is - minimal compared to the efforts necessary for an attacker to take a Tor - relay down. As a countermeasure to further frustrate this attack, the - hidden service raises the number of contact points as a function of - previous contact point failures. - - Further, the probability of breaking client connections due to attacking - a contact point is minimal. It can be assumed that the probability of one - of the other five involved relays in a hidden service connection failing - or being shut down is higher than that of a successful attack on a - contact point. - - (4) Resistance against Locating Attacks - - Clients are no longer able to force a hidden service to create or extend - circuits. This further reduces an attacker's capabilities of locating a - hidden server as described by Øverlier and Syverson [5]. - -Compatibility: - - The presented protocol does not raise compatibility issues with current - Tor versions. New relay versions support both, the existing and the - proposed protocol as introduction/rendezvous/contact points. A contact - point acts as introduction point simultaneously. Hidden services and - clients can opt-in to use the new protocol which might change to opt-out - some time in the future. - -References: - - [1] Roger Dingledine, Nick Mathewson, and Paul Syverson, Tor: The - Second-Generation Onion Router. In the Proceedings of the 13th USENIX - Security Symposium, August 2004. - - [2] Lasse Øverlier and Paul Syverson, Improving Efficiency and Simplicity - of Tor Circuit Establishment and Hidden Services. In the Proceedings of - the Seventh Workshop on Privacy Enhancing Technologies (PET 2007), - Ottawa, Canada, June 2007. - - [3] Christian Wilms, Improving the Tor Hidden Service Protocol Aiming at - Better Performance, diploma thesis, June 2008, University of Bamberg. - - [4] Lasse Øverlier and Paul Syverson, Valet Services: Improving Hidden - Servers with a Personal Touch. In the Proceedings of the Sixth Workshop - on Privacy Enhancing Technologies (PET 2006), Cambridge, UK, June 2006. - - [5] Lasse Øverlier and Paul Syverson, Locating Hidden Servers. In the - Proceedings of the 2006 IEEE Symposium on Security and Privacy, May 2006. - diff --git a/doc/spec/proposals/143-distributed-storage-improvements.txt b/doc/spec/proposals/143-distributed-storage-improvements.txt deleted file mode 100644 index 8789d8466..000000000 --- a/doc/spec/proposals/143-distributed-storage-improvements.txt +++ /dev/null @@ -1,196 +0,0 @@ -Filename: 143-distributed-storage-improvements.txt -Title: Improvements of Distributed Storage for Tor Hidden Service Descriptors -Version: $Revision$ -Last-Modified: $Date$ -Author: Karsten Loesing -Created: 28-Jun-2008 -Status: Open -Target: 0.2.1.x - -Change history: - - 28-Jun-2008 Initial proposal for or-dev - -Overview: - - An evaluation of the distributed storage for Tor hidden service - descriptors and subsequent discussions have brought up a few improvements - to proposal 114. All improvements are backwards compatible to the - implementation of proposal 114. - -Design: - - 1. Report Bad Directory Nodes - - Bad hidden service directory nodes could deny existence of previously - stored descriptors. A bad directory node that does this with all stored - descriptors causes harm to the distributed storage in general, but - replication will cope with this problem in most cases. However, an - adversary that attempts to make a specific hidden service unavailable by - running relays that become responsible for all of a service's - descriptors poses a more serious threat. The distributed storage needs to - defend against this attack by detecting and removing bad directory nodes. - - As a countermeasure hidden services try to download their descriptors - every hour at random times from the hidden service directories that are - responsible for storing it. If a directory node replies with 404 (Not - found), the hidden service reports the supposedly bad directory node to - a random selection of half of the directory authorities (with version - numbers equal to or higher than the first version that implements this - proposal). The hidden service posts a complaint message using HTTP 'POST' - to a URL "/tor/rendezvous/complain" with the following message format: - - "hidden-service-directory-complaint" identifier NL - - [At start, exactly once] - - The identifier of the hidden service directory node to be - investigated. - - "rendezvous-service-descriptor" descriptor NL - - [At end, Excatly once] - - The hidden service descriptor that the supposedly bad directory node - does not serve. - - The directory authority checks if the descriptor is valid and the hidden - service directory responsible for storing it. It waits for a random time - of up to 30 minutes before posting the descriptor to the hidden service - directory. If the publication is acknowledged, the directory authority - waits another random time of up to 30 minutes before attempting to - request the descriptor that it has posted. If the directory node replies - with 404 (Not found), it will be blacklisted for being a hidden service - directory node for the next 48 hours. - - A blacklisted hidden service directory is assigned the new flag BadHSDir - instead of the HSDir flag in the vote that a directory authority creates. - In a consensus a relay is only assigned a HSDir flag if the majority of - votes contains a HSDir flag and no more than one third of votes contains - a BadHSDir flag. As a result, clients do not have to learn about the - BadHSDir flag. A blacklisted directory node will simply not be assigned - the HSDir flag in the consensus. - - In order to prevent an attacker from setting up new nodes as replacement - for blacklisted directory nodes, all directory nodes in the same /24 - subnet are blacklisted, too. Furthermore, if two or more directory nodes - are blacklisted in the same /16 subnet concurrently, all other directory - nodes in that /16 subnet are blacklisted, too. Blacklisting holds for at - most 48 hours. - - 2. Publish Fewer Replicas - - The evaluation has shown that the probability of a directory node to - serve a previously stored descriptor is 85.7% (more precisely, this is - the 0.001-quantile of the empirical distribution with the rationale that - it holds for 99.9% of all empirical cases). If descriptors are replicated - to x directory nodes, the probability of at least one of the replicas to - be available for clients is 1 - (1 - 85.7%) ^ x. In order to achieve an - overall availability of 99.9%, x = 3.55 replicas need to be stored. From - this follows that 4 replicas are sufficient, rather than the currently - stored 6 replicas. - - Further, the current design stores 2 sets of descriptors on 3 directory - nodes with consecutive identities. Originally, this was meant to - facilitate replication between directory nodes, which has not been and - will not be implemented (the selection criterion of 24 hours uptime does - not make it necessary). As a result, storing descriptors on directory - nodes with consecutive identities is not required. In fact it should be - avoided to enable an attacker to create "black holes" in the identifier - ring. - - Hidden services should store their descriptors on 4 non-consecutive - directory nodes, and clients should request descriptors from these - directory nodes only. For compatibility reasons, hidden services also - store their descriptors on 2 consecutive directory nodes. Hence, 0.2.0.x - clients will be able to retrieve 4 out of 6 descriptors, but will fail - for the remaining 2 descriptors, which is sufficient for reliability. As - soon as 0.2.0.x is deprecated, hidden services can stop publishing the - additional 2 replicas. - - 3. Change Default Value of Being Hidden Service Directory - - The requirements for becoming a hidden service directory node are an open - directory port and an uptime of at least 24 hours. The evaluation has - shown that there are 300 hidden service directory candidates in the mean, - but only 6 of them are configured to act as hidden service directories. - This is bad, because those 6 nodes need to serve a large share of all - hidden service descriptors. Optimally, there should be hundreds of hidden - service directories. Having a large number of 0.2.1.x directory nodes - also has a positive effect on 0.2.0.x hidden services and clients. - - Therefore, the new default of HidServDirectoryV2 should be 1, so that a - Tor relay that has an open directory port automatically accepts and - serves v2 hidden service descriptors. A relay operator can still opt-out - running a hidden service directory by changing HidServDirectoryV2 to 0. - The additional bandwidth requirements for running a hidden service - directory node in addition to being a directory cache are negligible. - - 4. Make Descriptors Persistent on Directory Nodes - - Hidden service directories that are restarted by their operators or after - a failure will not be selected as hidden service directories within the - next 24 hours. However, some clients might still think that these nodes - are responsible for certain descriptors, because they work on the basis - of network consensuses that are up to three hours old. The directory - nodes should be able to serve the previously received descriptors to - these clients. Therefore, directory nodes make all received descriptors - persistent and load previously received descriptors on startup. - - 5. Store and Serve Descriptors Regardless of Responsibility - - Currently, directory nodes only accept descriptors for which they think - they are responsible. This may lead to problems when a directory node - uses an older or newer network consensus than hidden service or client - or when a directory node has been restarted recently. In fact, there are - no security issues in storing or serving descriptors for which a - directory node thinks it is not responsible. To the contrary, doing so - may improve reliability in border cases. As a result, a directory node - does not pay attention to responsibilty when receiving a publication or - fetch request, but stores or serves the requested descriptor. Likewise, - the directory node does not remove descriptors when it thinks it is not - responsible for them any more. - - 6. Avoid Periodic Descriptor Re-Publication - - In the current implementation a hidden service re-publishes its - descriptor either when its content changes or an hour elapses. However, - the evaluation has shown that failures of hidden service directory nodes, - i.e. of nodes that have not failed within the last 24 hours, are very - rare. Together with making descriptors persistent on directory nodes, - there is no necessity to re-publish descriptors hourly. - - The only two events leading to descriptor re-publication should be a - change of the descriptor content and a new directory node becoming - responsible for the descriptor. Hidden services should therefore consider - re-publication every time they learn about a new network consensus - instead of hourly. - - 7. Discard Expired Descriptors - - The current implementation lets directory nodes keep a descriptor for two - days before discarding it. However, with the v2 design, descriptors are - only valid for at most one day. Directory nodes should determine the - validity of stored descriptors and discard them one hour after they have - expired (to compensate wrong clocks on clients). - - 8. Shorten Client-Side Descriptor Fetch History - - When clients try to download a hidden service descriptor, they memorize - fetch requests to directory nodes for up to 15 minutes. This allows them - to request all replicas of a descriptor to avoid bad or failing directory - nodes, but without querying the same directory node twice. - - The downside is that a client that has requested a descriptor without - success, will not be able to find a hidden service that has been started - during the following 15 minutes after the client's last request. - - This can be improved by shortening the fetch history to only 5 minutes. - This time should be sufficient to complete requests for all replicas of a - descriptor, but without ending in an infinite request loop. - -Compatibility: - - All proposed improvements are compatible to the currently implemented - design as described in proposal 114. - diff --git a/doc/spec/proposals/144-enforce-distinct-providers.txt b/doc/spec/proposals/144-enforce-distinct-providers.txt deleted file mode 100644 index aa460482f..000000000 --- a/doc/spec/proposals/144-enforce-distinct-providers.txt +++ /dev/null @@ -1,165 +0,0 @@ -Filename: 144-enforce-distinct-providers.txt -Title: Increase the diversity of circuits by detecting nodes belonging the - same provider -Author: Mfr -Created: 2008-06-15 -Status: Draft - -Overview: - - Increase network security by reducing the capacity of the relay or - ISPs monitoring personally or requisition, a large part of traffic - Tor trying to break circuits privacy. A way to increase the - diversity of circuits without killing the network performance. - -Motivation: - - Since 2004, Roger an Nick publication about diversity [1], very fast - relays Tor running are focused among an half dozen of providers, - controlling traffic of some dozens of routers [2]. - - In the same way the generalization of VMs clonables paid by hour, - allowing starting in few minutes and for a small cost, a set of very - high-speed relay whose in a few hours can attract a big traffic that - can be analyzed, increasing the vulnerability of the network. - - Whether ISPs or domU providers, these usually have several groups of - IP Class B. Also the restriction in place EnforceDistinctSubnets - automatically excluding IP subnet class B is only partially - effective. By contrast a restriction at the class A will be too - restrictive. - - Therefore it seems necessary to consider another approach. - -Proposal: - - Add a provider control based on AS number added by the router on is - descriptor, controlled by Directories Authorities, and used like the - declarative family field for circuit creating. - -Design: - -Step 1 : - - Add to the router descriptor a provider information get request [4] - by the router itself. - - "provider" name NL - - 'names' is the AS number of the router formated like this: - 'ASxxxxxx' where AS is fixed and xxxxxx is the AS number, - left aligned ( ex: AS98304 , AS4096,AS1 ) or if AS number - is missing the network A class number is used like that: - 'ANxxx' where AN is fixed and xxx is the first 3 digits of - the IP (ex: for the IP 1.1.1.2 AN1) or an 'L' value is set - if it's a local network IP. - - If two ORs list one another in their "provider" entries, - then OPs should treat them as a single OR for the purpose - of path selection. - - For example, if node A's descriptor contains "provider B", - and node B's descriptor contains "provider A", then node A - and node B should never be used on the same circuit. - - Add the regarding config option in torrc - - EnforceDistinctProviders set to 1 by default. - Permit building circuits with relays in the same provider - if set to 0. - Regarding to proposal 135 if TestingTorNetwork is set - need to be EnforceDistinctProviders is unset. - - Control by Authorities Directories of the AS numbers - - The Directories Authority control the AS numbers of the new node - descriptor uploaded. - - If an old version is operated by the node this test is - bypassed. - - If AS number get by request is different from the - description, router is flagged as non-Valid by the testing - Authority for the voting process. - -Step 2 When a ' significant number of nodes' of valid routers are -generating descriptor with provider information. - - Add missing provider information get by DNS request -functionality for the circuit user: - - During circuit building, computing, OP apply first - family check and EnforceDistinctSubnets directives for - performance, then if provider info is needed and - missing in router descriptor try to get AS provider - info by DNS request [4]. This information could be - DNS cached. AN ( class A number) is never generated - during this process to prevent DNS block problems. If - DNS request fails ignore and continue building - circuit. - -Step 3 When the 'whole majority' of valid Tor clients are providing -DNS request. - - Older versions are deprecated and mark as no-Valid. - - EnforceDistinctProviders replace EnforceDistinctSubnets functionnality. - - EnforceDistinctSubnets is removed. - - Functionalities deployed in step 2 are removed. - -Security implications: - - This providermeasure will increase the number of providers - addresses that an attacker must use in order to carry out - traffic analysis. - -Compatibility: - - The presented protocol does not raise compatibility issues - with current Tor versions. The compatibility is preserved by - implementing this functionality in 3 steps, giving time to - network users to upgrade clients and routers. - -Performance and scalability notes: - - Provider change for all routers could reduce a little - performance if the circuit to long. - - During step 2 Get missing provider information could increase - building path time and should have a time out. - -Possible Attacks/Open Issues/Some thinking required: - - These proposal seems be compatible with proposal 135 Simplify - Configuration of Private Tor Networks. - - This proposal does not resolve multiples AS owners and top - providers traffic monitoring attacks [5]. - - Unresolved AS number are treated as a Class A network. Perhaps - should be marked as invalid. But there's only fives items on - last check see [2]. - - Need to define what's a 'significant number of nodes' and - 'whole majority' ;-) - -References: -[1] Location Diversity in Anonymity Networks by Nick Feamster and Roger -Dingledine. -In the Proceedings of the Workshop on Privacy in the Electronic Society -(WPES 2004), Washington, DC, USA, October 2004 -http://freehaven.net/anonbib/#feamster:wpes2004 -[2] http://as4jtw5gc6efb267.onion/IPListbyAS.txt -[3] see Goodell Tor Exit Page -http://cassandra.eecs.harvard.edu/cgi-bin/exit.py -[4] see the great IP to ASN DNS Tool -http://www.team-cymru.org/Services/ip-to-asn.html -[5] Sampled Traffic Analysis by Internet-Exchange-Level Adversaries by -Steven J. Murdoch and Piotr Zielinski. -In the Proceedings of the Seventh Workshop on Privacy Enhancing Technologies - -(PET 2007), Ottawa, Canada, June 2007. -http://freehaven.net/anonbib/#murdoch-pet2007 -[5] http://bugs.noreply.org/flyspray/index.php?do=details&id=690 diff --git a/doc/spec/proposals/145-newguard-flag.txt b/doc/spec/proposals/145-newguard-flag.txt deleted file mode 100644 index 31d707d72..000000000 --- a/doc/spec/proposals/145-newguard-flag.txt +++ /dev/null @@ -1,41 +0,0 @@ -Filename: 145-newguard-flag.txt -Title: Separate "suitable as a guard" from "suitable as a new guard" -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 1-Jul-2008 -Status: Open -Target: 0.2.1.x - -[This could be obsoleted by proposal 141, which could replace NewGuard -with a Guard weight.] - -Overview - - Right now, Tor has one flag that clients use both to tell which - nodes should be kept as guards, and which nodes should be picked - when choosing new guards. This proposal separates this flag into - two. - -Motivation - - Balancing clients amoung guards is not done well by our current - algorithm. When a new guard appears, it is chosen by clients - looking for a new guard with the same probability as all existing - guards... but new guards are likelier to be under capacity, whereas - old guards are likelier to be under more use. - -Implementation - - We add a new flag, NewGuard. Clients will change so that when they - are choosing new guards, they only consider nodes with the NewGuard - flag set. - - For now, authorities will always set NewGuard if they are setting - the Guard flag. Later, it will be easy to migrate authorities to - set NewGuard for underused guards. - -Alternatives - - We might instead have authorities list weights with which nodes - should be picked as guards. diff --git a/doc/spec/proposals/146-long-term-stability.txt b/doc/spec/proposals/146-long-term-stability.txt deleted file mode 100644 index 7cfd58f56..000000000 --- a/doc/spec/proposals/146-long-term-stability.txt +++ /dev/null @@ -1,86 +0,0 @@ -Filename: 146-long-term-stability.txt -Title: Add new flag to reflect long-term stability -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 19-Jun-2008 -Status: Open -Target: 0.2.1.x - -Overview - - This document proposes a new flag to indicate that a router has - existed at the same address for a long time, describes how to - implement it, and explains what it's good for. - -Motivation - - Tor has had three notions of "stability" for servers. Older - directory protocols based a server's stability on its - (self-reported) uptime: a server that had been running for a day was - more stable than a server that had been running for five minutes, - regardless of their past history. Current directory protocols track - weighted mean time between failure (WMTBF) and weighted fractional - uptime (WFU). WFU is computed as the fraction of time for which the - server is running, with measurements weighted to exponentially - decay such that old days count less. WMTBF is computed as the - average length of intervals for which the server runs between - downtime, with old intervals weighted to count less. - - WMTBF is useful in answering the question: "If a server is running - now, how long is it likely to stay running?" This makes it a good - choice for picking servers for streams that need to be long-lived. - WFU is useful in answering the question: "If I try connecting to - this server at an arbitrary time, is it likely to be running?" This - makes it an important factor for picking guard nodes, since we want - guard nodes to be usually-up. - - There are other questions that clients want to answer, however, for - which the current flags aren't very useful. The one that this - proposal addresses is, - - "If I found this server in an old consensus, is it likely to - still be running at the same address?" - - This one is useful when we're trying to find directory mirrors in a - fallback-consensus file. This property is equivalent to, - - "If I find this server in a current consensus, how long is it - likely to exist on the network?" - - This one is useful if we're trying to pick introduction points or - something and care more about churn rate than about whether every IP - will be up all the time. - -Implementation: - - I propose we add a new flag, called "Longterm." Authorities should - set this flag for routers if their Longevity is in the upper - quartile of all routers. A router's Longevity is computed as the - total amount of days in the last year or so[*] for which the router has - been Running at least once at its current IP:orport pair. - - Clients should use directory servers from a fallback-consensus only - if they have the Longterm flag set. - - Authority ops should be able to mark particular routers as not - Longterm, regardless of history. (For instance, it makes sense to - remove the Longterm flag from a router whose op says that it will - need to shutdown in a month.) - - [*] This is deliberately vague, to permit efficient implementations. - -Compatibility and migration issues: - - The voting protocol already acts gracefully when new flags are - added, so no change to the voting protocol is needed. - - Tor won't have collected this data, however. It might be desirable - to bootstrap it from historical consensuses. Alternatively, we can - just let the algorithm run for a month or two. - -Issues and future possibilities: - - Longterm is a really awkward name. - - diff --git a/doc/spec/proposals/147-prevoting-opinions.txt b/doc/spec/proposals/147-prevoting-opinions.txt deleted file mode 100644 index 2b8cf30e4..000000000 --- a/doc/spec/proposals/147-prevoting-opinions.txt +++ /dev/null @@ -1,60 +0,0 @@ -Filename: 147-prevoting-opinions.txt -Title: Eliminate the need for v2 directories in generating v3 directories -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 2-Jul-2008 -Status: Accepted -Target: 0.2.1.x - -Overview - - We propose a new v3 vote document type to replace the role of v2 - networkstatus information in generating v3 consensuses. - -Motivation - - When authorities vote on which descriptors are to be listed in the - next consensus, it helps if they all know about the same descriptors - as one another. But a hostile, confused, or out-of-date server may - upload a descriptor to only some authorities. In the current v3 - directory design, the authorities don't have a good way to tell one - another about the new descriptor until they exchange votes... but by - the time this happens, they are already committed to their votes, - and they can't add anybody they learn about from other authorities - until the next voting cycle. That's no good! - - The current Tor implementation avoids this problem by having - authorities also look at v2 networkstatus documents, but we'd like - in the long term to eliminate these, once 0.1.2.x is obsolete. - -Design: - - We add a new value for vote-status in v3 consensus documents in - addition to "consensus" and "vote": "opinion". Authorities generate - and sign an opinion document as if they were generating a vote, - except that they generate opinions earlier than they generate votes. - - Authorities don't need to generate more than one opinion document - per voting interval, but may. They should send it to the other - authorities they know about, at the regular vote upload URL, before - the authorities begin voting, so that enough time remains for the - authorities to fetch new descriptors. - - Additionally, authories make their opinions available at - http:///tor/status-vote/next/opinion.z - and download opinions from authorities they haven't heard from in a - while. - - Authorities MAY generate opinions on demand. - - Upon receiving an opinion document, authorities scan it for any - descriptors that: - - They might accept. - - Are for routers they don't know about, or are published more - recently than any descriptor they have for that router. - Authorities then begin downloading such descriptors from authorities - that claim to have them. - - Authorities MAY cache opinion documents, but don't need to. - diff --git a/doc/spec/proposals/148-uniform-client-end-reason.txt b/doc/spec/proposals/148-uniform-client-end-reason.txt deleted file mode 100644 index cec81253e..000000000 --- a/doc/spec/proposals/148-uniform-client-end-reason.txt +++ /dev/null @@ -1,59 +0,0 @@ -Filename: 148-uniform-client-end-reason.txt -Title: Stream end reasons from the client side should be uniform -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 2-Jul-2008 -Status: Closed -Implemented-In: 0.2.1.9-alpha - -Overview - - When a stream closes before it's finished, the end relay cell that's - sent includes an "end stream reason" to tell the other end why it - closed. It's useful for the exit relay to send a reason to the client, - so the client can choose a different circuit, inform the user, etc. But - there's no reason to include it from the client to the exit relay, - and in some cases it can even harm anonymity. - - We should pick a single reason for the client-to-exit-relay direction - and always just send that. - -Motivation - - Back when I first deployed the Tor network, it was useful to have - the Tor relays learn why a stream closed, so I could debug both ends - of the stream at once. Now that streams have worked for many years, - there's no need to continue telling the exit relay whether the client - gave up on a stream because of "timeout" or "misc" or what. - - Then in Tor 0.2.0.28-rc, I fixed this bug: - - Fix a bug where, when we were choosing the 'end stream reason' to - put in our relay end cell that we send to the exit relay, Tor - clients on Windows were sometimes sending the wrong 'reason'. The - anonymity problem is that exit relays may be able to guess whether - the client is running Windows, thus helping partition the anonymity - set. Down the road we should stop sending reasons to exit relays, - or otherwise prevent future versions of this bug. - - It turned out that non-Windows clients were choosing their reason - correctly, whereas Windows clients were potentially looking at errno - wrong and so always choosing 'misc'. - - I fixed that particular bug, but I think we should prevent future - versions of the bug too. - - (We already fixed it so *circuit* end reasons don't get sent from - the client to the exit relay. But we appear to be have skipped over - stream end reasons thus far.) - -Design: - - One option would be to no longer include any 'reason' field in end - relay cells. But that would introduce a partitioning attack ("users - running the old version" vs "users running the new version"). - - Instead I suggest that clients all switch to sending the "misc" reason, - like most of the Windows clients currently do and like the non-Windows - clients already do sometimes. - diff --git a/doc/spec/proposals/149-using-netinfo-data.txt b/doc/spec/proposals/149-using-netinfo-data.txt deleted file mode 100644 index 4919514b4..000000000 --- a/doc/spec/proposals/149-using-netinfo-data.txt +++ /dev/null @@ -1,44 +0,0 @@ -Filename: 149-using-netinfo-data.txt -Title: Using data from NETINFO cells -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 2-Jul-2008 -Status: Open -Target: 0.2.1.x - -Overview - - Current Tor versions send signed IP and timestamp information in - NETINFO cells, but don't use them to their fullest. This proposal - describes how they should start using this info in 0.2.1.x. - -Motivation - - Our directory system relies on clients and routers having - reasonably accurate clocks to detect replayed directory info, and - to set accurate timestamps on directory info they publish - themselves. NETINFO cells contain timestamps. - - Also, the directory system relies on routers having a reasonable - idea of their own IP addresses, so they can publish correct - descriptors. This is also in NETINFO cells. - -Learning the time and IP - - We need to think about attackers here. Just because a router tells - us that we have a given IP or a given clock skew doesn't mean that - it's true. We believe this information only if we've heard it from - a majority of the routers we've connected to recently, including at - least 3 routers. Routers only believe this information if the - majority inclues at least one authority. - -Avoiding MITM attacks - - Current Tors use the IP addresses published in the other router's - NETINFO cells to see whether the connection is "canonical". Right - now, we prefer to extend circuits over "canonical" connections. In - 0.2.1.x, we should refuse to extend circuits over non-canonical - connections without first trying to build a canonical one. - - diff --git a/doc/spec/proposals/150-exclude-exit-nodes.txt b/doc/spec/proposals/150-exclude-exit-nodes.txt deleted file mode 100644 index b73a9cc4d..000000000 --- a/doc/spec/proposals/150-exclude-exit-nodes.txt +++ /dev/null @@ -1,48 +0,0 @@ -Filename: 150-exclude-exit-nodes.txt -Title: Exclude Exit Nodes from a circuit -Version: $Revision$ -Author: Mfr -Created: 2008-06-15 -Status: Closed -Implemented-In: 0.2.1.3-alpha - -Overview - - Right now, Tor users can manually exclude a node from all positions - in their circuits created using the directive ExcludeNodes. - This proposal makes this exclusion less restrictive, allowing users to - exclude a node only from the exit part of a circuit. - -Motivation - - This feature would Help the integration into vidalia (tor exit - branch) or other tools, of features to exclude a country for exit - without reducing circuits possibilities, and privacy. This feature - could help people from a country were many sites are blocked to - exclude this country for browsing, giving them a more stable - navigation. It could also add the possibility for the user to - exclude a currently used exit node. - -Implementation - - ExcludeExitNodes is similar to ExcludeNodes except it's only - the exit node which is excluded for circuit build. - - Tor doesn't warn if node from this list is not an exit node. - -Security implications: - - Open also possibilities for a future user bad exit reporting - -Risks: - - Use of this option can make users partitionable under certain attack - assumptions. However, ExitNodes already creates this possibility, - so there isn't much increased risk in ExcludeExitNodes. - - We should still encourage people who exclude an exit node because - of bad behavior to report it instead of just adding it to their - ExcludeExit list. It would be unfortunate if we didn't find out - about broken exits because of this option. This issue can probably - be addressed sufficiently with documentation. - diff --git a/doc/spec/proposals/151-path-selection-improvements.txt b/doc/spec/proposals/151-path-selection-improvements.txt deleted file mode 100644 index e3c8f3545..000000000 --- a/doc/spec/proposals/151-path-selection-improvements.txt +++ /dev/null @@ -1,147 +0,0 @@ -Filename: 151-path-selection-improvements.txt -Title: Improving Tor Path Selection -Version: -Last-Modified: -Author: Fallon Chen, Mike Perry -Created: 5-Jul-2008 -Status: Draft - -Overview - - The performance of paths selected can be improved by adjusting the - CircuitBuildTimeout and avoiding failing guard nodes. This proposal - describes a method of tracking buildtime statistics at the client, and - using those statistics to adjust the CircuitBuildTimeout. - -Motivation - - Tor's performance can be improved by excluding those circuits that - have long buildtimes (and by extension, high latency). For those Tor - users who require better performance and have lower requirements for - anonymity, this would be a very useful option to have. - -Implementation - - Storing Build Times - - Circuit build times will be stored in the circular array - 'circuit_build_times' consisting of uint16_t elements as milliseconds. - The total size of this array will be based on the number of circuits - it takes to converge on a good fit of the long term distribution of - the circuit builds for a fixed link. We do not want this value to be - too large, because it will make it difficult for clients to adapt to - moving between different links. - - From our initial observations, this value appears to be on the order - of 1000, but will be configurable in a #define NCIRCUITS_TO_OBSERVE. - The exact value for this #define will be determined by performing - goodness of fit tests using measurments obtained from the shufflebt.py - script from TorFlow. - - Long Term Storage - - The long-term storage representation will be implemented by storing a - histogram with BUILDTIME_BIN_WIDTH millisecond buckets (default 50) when - writing out the statistics to disk. The format of this histogram on disk - is yet to be finalized, but it will likely be of the format - 'CircuitBuildTime ', with the total specified as - 'TotalBuildTimes ' - Example: - - TotalBuildTimes 100 - CircuitBuildTimeBin 1 50 - CircuitBuildTimeBin 2 25 - CircuitBuildTimeBin 3 13 - ... - - Reading the histogram in will entail multiplying each bin by the - BUILDTIME_BIN_WIDTH and then inserting values into the - circuit_build_times array each with the value of - *BUILDTIME_BIN_WIDTH. In order to evenly distribute the - values in the circular array, a form of index skipping must - be employed. Values from bin #N with bin count C and total T - will occupy indexes specified by N+((T/C)*k)-1, where k is the - set of integers ranging from 0 to C-1. - - For example, this would mean that the values from bin 1 would - occupy indexes 1+(100/50)*k-1, or 0, 2, 4, 6, 8, 10 and so on. - The values for bin 2 would occupy positions 1, 5, 9, 13. Collisions - will be inserted at the first empty position in the array greater - than the selected index (which may requiring looping around the - array back to index 0). - - Learning the CircuitBuildTimeout - - Based on studies of build times, we found that the distribution of - circuit buildtimes appears to be a Pareto distribution. - - We will calculate the parameters for a Pareto distribution - fitting the data using the estimators at - http://en.wikipedia.org/wiki/Pareto_distribution#Parameter_estimation. - - The timeout itself will be calculated by solving the CDF for the - a percentile cutoff BUILDTIME_PERCENT_CUTOFF. This value - represents the percentage of paths the Tor client will accept out of - the total number of paths. We have not yet determined a good - cutoff for this mathematically, but 85% seems a good choice for now. - - From http://en.wikipedia.org/wiki/Pareto_distribution#Definition, - the calculation we need is pow(BUILDTIME_PERCENT_CUTOFF/100.0, k)/Xm. - - Testing - - After circuit build times, storage, and learning are implemented, - the resulting histogram should be checked for consistency by - verifying it persists across successive Tor invocations where - no circuits are built. In addition, we can also use the existing - buildtime scripts to record build times, and verify that the histogram - the python produces matches that which is output to the state file in Tor, - and verify that the Pareto parameters and cutoff points also match. - - Soft timeout vs Hard Timeout - - At some point, it may be desirable to change the cutoff from a - single hard cutoff that destroys the circuit to a soft cutoff and - a hard cutoff, where the soft cutoff merely triggers the building - of a new circuit, and the hard cutoff triggers destruction of the - circuit. - - Good values for hard and soft cutoffs seem to be 85% and 65% - respectively, but we should eventually justify this with observation. - - When to Begin Calculation - - The number of circuits to observe (NCIRCUITS_TO_CUTOFF) before - changing the CircuitBuildTimeout will be tunable via a #define. From - our measurements, a good value for NCIRCUITS_TO_CUTOFF appears to be - on the order of 100. - - Dealing with Timeouts - - Timeouts should be counted as the expectation of the region of - of the Pareto distribution beyond the cutoff. The proposal will - be updated with this value soon. - - Also, in the event of network failure, the observation mechanism - should stop collecting timeout data. - - Client Hints - - Some research still needs to be done to provide initial values - for CircuitBuildTimeout based on values learned from modem - users, DSL users, Cable Modem users, and dedicated links. A - radiobutton in Vidalia should eventually be provided that - sets CircuitBuildTimeout to one of these values and also - provide the option of purging all learned data, should any exist. - - These values can either be published in the directory, or - shipped hardcoded for a particular Tor version. - -Issues - - Impact on anonymity - - Since this follows a Pareto distribution, large reductions on the - timeout can be achieved without cutting off a great number of the - total paths. This will eliminate a great deal of the performance - variation of Tor usage. diff --git a/doc/spec/proposals/152-single-hop-circuits.txt b/doc/spec/proposals/152-single-hop-circuits.txt deleted file mode 100644 index e49a4250e..000000000 --- a/doc/spec/proposals/152-single-hop-circuits.txt +++ /dev/null @@ -1,64 +0,0 @@ -Filename: 152-single-hop-circuits.txt -Title: Optionally allow exit from single-hop circuits -Version: -Last-Modified: -Author: Geoff Goodell -Created: 13-Jul-2008 -Status: Closed -Implemented-In: 0.2.1.6-alpha - -Overview - - Provide a special configuration option that adds a line to descriptors - indicating that a router can be used as an exit for one-hop circuits, - and allow clients to attach streams to one-hop circuits provided - that the descriptor for the router in the circuit includes this - configuration option. - -Motivation - - At some point, code was added to restrict the attachment of streams - to one-hop circuits. - - The idea seems to be that we can use the cost of forking and - maintaining a patch as a lever to prevent people from writing - controllers that jeopardize the operational security of routers - and the anonymity properties of the Tor network by creating and - using one-hop circuits rather than the standard three-hop circuits. - It may be, for example, that some users do not actually seek true - anonymity but simply reachability through network perspectives - afforded by the Tor network, and since anonymity is stronger in - numbers, forcing users to contribute to anonymity and decrease the - risk to server operators by using full-length paths may be reasonable. - - As presently implemented, the sweeping restriction of one-hop circuits - for all routers limits the usefulness of Tor as a general-purpose - technology for building circuits. In particular, we should allow - for controllers, such as Blossom, that create and use single-hop - circuits involving routers that are not part of the Tor network. - -Design - - Introduce a configuration option for Tor servers that, when set, - indicates that a router is willing to provide exit from one-hop - circuits. Routers with this policy will not require that a circuit - has at least two hops when it is used as an exit. - - In addition, routers for which this configuration option - has been set will have a line in their descriptors, "opt - exit-from-single-hop-circuits". Clients will keep track of which - routers have this option and allow streams to be attached to - single-hop circuits that include such routers. - -Security Considerations - - This approach seems to eliminate the worry about operational router - security, since server operators will not set the configuraiton - option unless they are willing to take on such risk. - - To reduce the impact on anonymity of the network resulting - from including such "risky" routers in regular Tor path - selection, clients may systematically exclude routers with "opt - exit-from-single-hop-circuits" when choosing random paths through - the Tor network. - diff --git a/doc/spec/proposals/153-automatic-software-update-protocol.txt b/doc/spec/proposals/153-automatic-software-update-protocol.txt deleted file mode 100644 index 7bc809d44..000000000 --- a/doc/spec/proposals/153-automatic-software-update-protocol.txt +++ /dev/null @@ -1,177 +0,0 @@ -Filename: 153-automatic-software-update-protocol.txt -Title: Automatic software update protocol -Version: $Revision$ -Last-Modified: $Date$ -Author: Jacob Appelbaum -Created: 14-July-2008 -Status: Superseded - -[Superseded by thandy-spec.txt] - - - Automatic Software Update Protocol Proposal - -0.0 Introduction - -The Tor project and its users require a robust method to update shipped -software bundles. The software bundles often includes Vidalia, Privoxy, Polipo, -Torbutton and of course Tor itself. It is not inconcievable that an update -could include all of the Tor Browser Bundle. It seems reasonable to make this -a standalone program that can be called in shell scripts, cronjobs or by -various Tor controllers. - -0.1 Minimal Tasks To Implement Automatic Updating - -At the most minimal, an update must be able to do the following: - - 0 - Detect the curent Tor version, note the working status of Tor. - 1 - Detect the latest Tor version. - 2 - Fetch the latest version in the form of a platform specific package(s). - 3 - Verify the itegrity of the downloaded package(s). - 4 - Install the verified package(s). - 5 - Test that the new package(s) works properly. - -0.2 Specific Enumeration Of Minimal Tasks - -To implement requirement 0, we need to detect the current Tor version of both -the updater and the current running Tor. The update program itself should be -versioned internally. This requirement should also test connecting through Tor -itself and note if such connections are possible. - -To implement requirement 1, we need to learn the concensus from the directory -authorities or fail back to a known good URL with cryptographically signed -content. - -To implement requirement 2, we need to download Tor - hopefully over Tor. - -To implement requirement 3, we need to verify the package signature. - -To implement requirement 4, we need to use a platform specific method of -installation. The Tor controller performing the update perform these platform -specific methods. - -To implement requirement 5, we need to be able to extend circuits and reach -the internet through Tor. - -0.x Implementation Goals - -The update system will be cross platform and rely on as little external code -as possible. If the update system uses it, it must be updated by the update -system itself. It will consist only of free software and will not rely on any -non-free components until the actual installation phase. If a package manager -is in use, it will be platform specific and thus only invoked by the update -system implementing the update protocol. - -The update system itself will attempt to perform update related network -activity over Tor. Possibly it will attempt to use a hidden service first. -It will attempt to use novel and not so novel caching -when possible, it will always verify cryptographic signatures before any -remotely fetched code is executed. In the event of an unusable Tor system, -it will be able to attempt to fetch updates without Tor. This should be user -configurable, some users will be unwilling to update without the protection of -using Tor - others will simply be unable because of blocking of the main Tor -website. - -The update system will track current version numbers of Tor and supporting -software. The update system will also track known working versions to assist -with automatic The update system itself will be a standalone library. It will be -strongly versioned internally to match the Tor bundle it was shiped with. The -update system will keep track of the given platform, cpu architecture, lsb_release, -package management functionality and any other platform specific metadata. - -We have referenced two popular automatic update systems, though neither fit -our needs, both are useful as an idea of what others are doing in the same -area. - -The first is sparkle[0] but it is sadly only available for Cocoa -environments and is written in Objective C. This doesn't meet our requirements -because it is directly tied into the private Apple framework. - -The second is the Mozilla Automatic Update System[1]. It is possibly useful -as an idea of how other free software projects automatically update. It is -however not useful in its currently documented form. - - - [0] http://sparkle.andymatuschak.org/documentation/ - [1] http://wiki.mozilla.org/AUS:Manual - -0.x Previous methods of Tor and related software update - -Previously, Tor users updated their Tor related software by hand. There has -been no fully automatic method for any user to update. In addition, there -hasn't been any specific way to find out the most current stable version of Tor -or related software as voted on by the directory authority concensus. - -0.x Changes to the directory specification - -We will want to supplement client-versions and server-versions in the -concensus voting with another version identifier known as -'auto-update-versions'. This will keep track of the current concensus of -specific versions that are best per platform and per architecture. It should -be noted that while the Mac OS X universal binary may be the best for x86 -processers with Tiger, it may not be the best for PPC users on Panther. This -goes for all of the package updates. We want to prevent updates that cause Tor -to break even if the updating program can recover gracefully. - -x.x Assumptions About Operating System Package Management - -It is assumed that users will use their package manager unless they are on -Microsoft Windows (any version) or Mac OS X (any version). Microsoft Windows -users will have integration with the normal "add/remove program" functionality -that said users would expect. - -x.x Package Update System Failure Modes - -The package update will try to ensure that a user always has a working Tor at -the very least. It will keep state to remember versions of Tor that were able -to bootstrap properly and reach the rest of the Tor network. It will also keep -note of which versions broke. It will select the best Tor that works for the -user. It will also allow for anonymized bug reporting on the packages -available and tested by the auto-update system. - -x.x Package Signature Verification - -The update system will be aware of replay attacks against the update signature -system itself. It will not allow package update signatures that are radically -out of date. It will be a multi-key system to prevent any single party from -forging an update. The key will be updated regularly. This is like authority -key (see proposal 103) usage. - -x.x Package Caching - -The update system will iterate over different update methods. Whichever method -is picked will have caching functionality. Each Tor server itself should be -able to serve cached update files. This will be an option that friendly server -administrators can turn on should they wish to support caching. In addition, -it is possible to cache the full contents of a package in an -authoratative DNS zone. Users can then query the DNS zone for their package. -If we wish to further distribute the update load, we can also offer packages -with encrypted bittorrent. Clients who wish to share the updates but do not -wish to be a server can help distribute Tor updates. This can be tied together -with the DNS caching[2][3] if needed. - - [2] http://www.netrogenic.com/dnstorrent/ - [3] http://www.doxpara.com/ozymandns_src_0.1.tgz - -x.x Helping Our Users Spread Tor - -There should be a way for a user to participate in the packaging caching as -described in section x.x. This option should be presented by the Tor -controller. - -x.x Simple HTTP Proxy To The Tor Project Website - -It has been suggested that we should provide a simple proxy that allows a user -to visit the main Tor website to download packages. This was part of a -previous proposal and has not been closely examined. - -x.x Package Installation - -Platform specific methods for proper package installation will be left to the -controller that is calling for an update. Each platform is different, the -installation options and user interface will be specific to the controller in -question. - -x.x Other Things - -Other things should be added to this proposal. What are they? diff --git a/doc/spec/proposals/154-automatic-updates.txt b/doc/spec/proposals/154-automatic-updates.txt deleted file mode 100644 index 00a820de0..000000000 --- a/doc/spec/proposals/154-automatic-updates.txt +++ /dev/null @@ -1,379 +0,0 @@ -Filename: 154-automatic-updates.txt -Title: Automatic Software Update Protocol -Version: $Revision$ -Last-Modified: $Date$ -Author: Matt Edman -Created: 30-July-2008 -Status: Superseded -Target: 0.2.1.x - -Superseded by thandy-spec.txt - -Scope - - This proposal specifies the method by which an automatic update client can - determine the most recent recommended Tor installation package for the - user's platform, download the package, and then verify that the package was - downloaded successfully. While this proposal focuses on only the Tor - software, the protocol defined is sufficiently extensible such that other - components of the Tor bundles, like Vidalia, Polipo, and Torbutton, can be - managed and updated by the automatic update client as well. - - The initial target platform for the automatic update framework is Windows, - given that's the platform used by a majority of our users and that it lacks - a sane package management system that many Linux distributions already have. - Our second target platform will be Mac OS X, and so the protocol will be - designed with this near-future direction in mind. - - Other client-side aspects of the automatic update process, such as user - interaction, the interface presented, and actual package installation - procedure, are outside the scope of this proposal. - - -Motivation - - Tor releases new versions frequently, often with important security, - anonymity, and stability fixes. Thus, it is important for users to be able - to promptly recognize when new versions are available and to easily - download, authenticate, and install updated Tor and Tor-related software - packages. - - Tor's control protocol [2] provides a method by which controllers can - identify when the user's Tor software is obsolete or otherwise no longer - recommended. Currently, however, no mechanism exists for clients to - automatically download and install updated Tor and Tor-related software for - the user. - - -Design Overview - - The core of the automatic update framework is a well-defined file called a - "recommended-packages" file. The recommended-packages file is accessible via - HTTP[S] at one or more well-defined URLs. An example recommended-packages - URL may be: - - https://updates.torproject.org/recommended-packages - - The recommended-packages document is formatted according to Section 1.2 - below and specifies the most recent recommended installation package - versions for Tor or Tor-related software, as well as URLs at which the - packages and their signatures can be downloaded. - - An automatic update client process runs on the Tor user's computer and - periodically retrieves the recommended-packages file according to the method - described in Section 2.0. As described further in Section 1.2, the - recommended-packages file is signed and can be verified by the automatic - update client with one or more public keys included in the client software. - Since it is signed, the recommended-packages file can be mirrored by - multiple hosts (e.g., Tor directory authorities), whose URLs are included in - the automatic update client's configuration. - - After retrieving and verifying the recommended-packages file, the automatic - update client compares the versions of the recommended software packages - listed in the file with those currently installed on the end-user's - computer. If one or more of the installed packages is determined to be out - of date, an updated package and its signature will be downloaded from one of - the package URLs listed in the recommended-packages file as described in - Section 2.2. - - The automatic update system uses a multilevel signing key scheme for package - signatures. There are a small number of entities we call "packaging - authorities" that each have their own signing key. A packaging authority is - responsible for signing and publishing the recommended-packages file. - Additionally, each individual packager responsible for producing an - installation package for one or more platforms has their own signing key. - Every packager's signing key must be signed by at least one of the packaging - authority keys. - - -Specification - - 1. recommended-packages Specification - - In this section we formally specify the format of the published - recommended-packages file. - - 1.1. Document Meta-format - - The recommended-packages document follows the lightweight extensible - information format defined in Tor's directory protocol specification [1]. In - the interest of self-containment, we have reproduced the relevant portions - of that format's specification in this Section. (Credits to Nick Mathewson - for much of the original format definition language.) - - The highest level object is a Document, which consists of one or more - Items. Every Item begins with a KeywordLine, followed by zero or more - Objects. A KeywordLine begins with a Keyword, optionally followed by - whitespace and more non-newline characters, and ends with a newline. A - Keyword is a sequence of one or more characters in the set [A-Za-z0-9-]. - An Object is a block of encoded data in pseudo-Open-PGP-style - armor. (cf. RFC 2440) - - More formally: - - Document ::= (Item | NL)+ - Item ::= KeywordLine Object* - KeywordLine ::= Keyword NL | Keyword WS ArgumentChar+ NL - Keyword ::= KeywordChar+ - KeywordChar ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-' - ArgumentChar ::= any printing ASCII character except NL. - WS ::= (SP | TAB)+ - Object ::= BeginLine Base-64-encoded-data EndLine - BeginLine ::= "-----BEGIN " Keyword "-----" NL - EndLine ::= "-----END " Keyword "-----" NL - - The BeginLine and EndLine of an Object must use the same keyword. - - In our Document description below, we also tag Items with a multiplicity in - brackets. Possible tags are: - - "At start, exactly once": These items MUST occur in every instance of the - document type, and MUST appear exactly once, and MUST be the first item in - their documents. - - "Exactly once": These items MUST occur exactly one time in every - instance of the document type. - - "Once or more": These items MUST occur at least once in any instance - of the document type, and MAY occur more than once. - - "At end, exactly once": These items MUST occur in every instance of - the document type, and MUST appear exactly once, and MUST be the - last item in their documents. - - 1.2. recommended-packages Document Format - - When interpreting a recommended-packages Document, software MUST ignore - any KeywordLine that starts with a keyword it doesn't recognize; future - implementations MUST NOT require current automatic update clients to - understand any KeywordLine not currently described. - - In lines that take multiple arguments, extra arguments SHOULD be - accepted and ignored. - - The currently defined Items contained in a recommended-packages document - are: - - "recommended-packages-format" SP number NL - - [Exactly once] - - This Item specifies the version of the recommended-packages format that - is contained in the subsequent document. The version defined in this - proposal is version "1". Subsequent iterations of this protocol MUST - increment this value if they introduce incompatible changes to the - document format and MAY increment this value if they only introduce - additional Keywords. - - "published" SP YYYY-MM-DD SP HH:MM:SS NL - - [Exactly once] - - The time, in GMT, when this recommended-packages document was generated. - Automatic update clients SHOULD ignore Documents over 60 days old. - - "tor-stable-win32-version" SP TorVersion NL - - [Exactly once] - - This keyword specifies the latest recommended release of Tor's "stable" - branch for the Windows platform that has an installation package - available. Note that this version does not necessarily correspond to the - most recently tagged stable Tor version, since that version may not yet - have an installer package available, or may have known issues on - Windows. - - The TorVersion field is formatted according to Section 2 of Tor's - version specification [3]. - - "tor-stable-win32-package" SP Url NL - - [Once or more] - - This Item specifies the location from which the most recent - recommended Windows installation package for Tor's stable branch can be - downloaded. - - When this Item appears multiple times within the Document, automatic - update clients SHOULD select randomly from the available package - mirrors. - - "tor-dev-win32-version" SP TorVersion NL - - [Exactly once] - - This Item specifies the latest recommended release of Tor's - "development" branch for the Windows platform that has an installation - package available. The same caveats from the description of - "tor-stable-win32-version" also apply to this keyword. - - The TorVersion field is formatted according to Section 2 of Tor's - version specification [3]. - - "tor-dev-win32-package" SP Url NL - - [Once or more] - - This Item specifies the location from which the most recent recommended - Windows installation package and its signature for Tor's development - branch can be downloaded. - - When this Keyword appears multiple times within the Document, automatic - update clients SHOULD select randomly from the available package - mirrors. - - "signature" NL SIGNATURE NL - - [At end, exactly once] - - The "SIGNATURE" Object contains a PGP signature (using a packaging - authority signing key) of the entire document, taken from the beginning - of the "recommended-packages-format" keyword, through the newline after - the "signature" Keyword. - - - 2. Automatic Update Client Behavior - - The client-side component of the automatic update framework is an - application that runs on the end-user's machine. It is responsible for - fetching and verifying a recommended-packages document, as well as - downloading, verifying, and subsequently installing any necessary updated - software packages. - - 2.1. Download and verify a recommended-packages document - - The first step in the automatic update process is for the client to download - a copy of the recommended-packages file. The automatic update client - contains a (hardcoded and/or user-configurable) list of URLs from which it - will attempt to retrieve a recommended-packages file. - - Connections to each of the recommended-packages URLs SHOULD be attempted in - the following order: - - 1) HTTPS over Tor - 2) HTTP over Tor - 3) Direct HTTPS - 4) Direct HTTP - - If the client fails to retrieve a recommended-packages document via any of - the above connection methods from any of the configured URLs, the client - SHOULD retry its download attempts following an exponential back-off - algorithm. After the first failed attempt, the client SHOULD delay one hour - before attempting again, up to a maximum of 24 hours delay between retry - attempts. - - After successfully downloading a recommended-packages file, the automatic - update client will verify the signature using one of the public keys - distributed with the client software. If more than one recommended-packages - file is downloaded and verified, the file with the most recent "published" - date that is verified will be retained and the rest discarded. - - 2.2. Download and verify the updated packages - - The automatic update client next compares the latest recommended package - version from the recommended-packages document with the currently installed - Tor version. If the user currently has installed a Tor version from Tor's - "development" branch, then the version specified in "tor-dev-*-version" Item - is used for comparison. Similarly, if the user currently has installed a Tor - version from Tor's "stable" branch, then the version specified in the - "tor-stable-*version" Item is used for comparison. Version comparisons are - done according to Tor's version specification [3]. - - If the automatic update client determines an installation package newer than - the user's currently installed version is available, it will attempt to - download a package appropriate for the user's platform and Tor branch from a - URL specified by a "tor-[branch]-[platform]-package" Item. If more than one - mirror for the selected package is available, a mirror will be chosen at - random from all those available. - - The automatic update client must also download a ".asc" signature file for - the retrieved package. The URL for the package signature is the same as that - for the package itself, except with the extension ".asc" appended to the - package URL. - - Connections to download the updated package and its signature SHOULD be - attempted in the same order described in Section 2.1. - - After completing the steps described in Sections 2.1 and 2.2, the automatic - update client will have downloaded and verified a copy of the latest Tor - installation package. It can then take whatever subsequent platform-specific - steps are necessary to install the downloaded software updates. - - 2.3. Periodic checking for updates - - The automatic update client SHOULD maintain a local state file in which it - records (at a minimum) the timestamp at which it last retrieved a - recommended-packages file and the timestamp at which the client last - successfully downloaded and installed a software update. - - Automatic update clients SHOULD check for an updated recommended-packages - document at most once per day but at least once every 30 days. - - - 3. Future Extensions - - There are several possible areas for future extensions of this framework. - The extensions below are merely suggestions and should be the subject of - their own proposal before being implemented. - - 3.1. Additional Software Updates - - There are several software packages often included in Tor bundles besides - Tor, such as Vidalia, Privoxy or Polipo, and Torbutton. The versions and - download locations of updated installation packages for these bundle - components can be easily added to the recommended-packages document - specification above. - - 3.2. Including ChangeLog Information - - It may be useful for automatic update clients to be able to display for - users a summary of the changes made in the latest Tor or Tor-related - software release, before the user chooses to install the update. In the - future, we can add keywords to the specification in Section 1.2 that specify - the location of a ChangeLog file for the latest recommended package - versions. It may also be desirable to allow localized ChangeLog information, - so that the automatic update client can fetch release notes in the - end-user's preferred language. - - 3.3. Weighted Package Mirror Selection - - We defined in Section 1.2 a method by which automatic update clients can - select from multiple available package mirrors. We may want to add a Weight - argument to the "*-package" Items that allows the recommended-packages file - to suggest to clients the probability with which a package mirror should be - chosen. This will allow clients to more appropriately distribute package - downloads across available mirrors proportional to their approximate - bandwidth. - - -Implementation - - Implementation of this proposal will consist of two separate components. - - The first component is a small "au-publish" tool that takes as input a - configuration file specifying the information described in Section 1.2 and a - private key. The tool is run by a "packaging authority" (someone responsible - for publishing updated installation packages), who will be prompted to enter - the passphrase for the private key used to sign the recommended-packages - document. The output of the tool is a document formatted according to - Section 1.2, with a signature appended at the end. The resulting document - can then be published to any of the update mirrors. - - The second component is an "au-client" tool that is run on the end-user's - machine. It periodically checks for updated installation packages according - to Section 2 and fetches the packages if necessary. The public keys used - to sign the recommended-packages file and any of the published packages are - included in the "au-client" tool. - - -References - - [1] Tor directory protocol (version 3), - https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/dir-spec.txt - - [2] Tor control protocol (version 2), - https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/control-spec.txt - - [3] Tor version specification, - https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/version-spec.txt - diff --git a/doc/spec/proposals/155-four-hidden-service-improvements.txt b/doc/spec/proposals/155-four-hidden-service-improvements.txt deleted file mode 100644 index f528f8baf..000000000 --- a/doc/spec/proposals/155-four-hidden-service-improvements.txt +++ /dev/null @@ -1,122 +0,0 @@ -Filename: 155-four-hidden-service-improvements.txt -Title: Four Improvements of Hidden Service Performance -Version: $Revision$ -Last-Modified: $Date$ -Author: Karsten Loesing, Christian Wilms -Created: 25-Sep-2008 -Status: Finished -Implemented-In: 0.2.1.x - -Change history: - - 25-Sep-2008 Initial proposal for or-dev - -Overview: - - A performance analysis of hidden services [1] has brought up a few - possible design changes to reduce advertisement time of a hidden service - in the network as well as connection establishment time. Some of these - design changes have side-effects on anonymity or overall network load - which had to be weighed up against individual performance gains. A - discussion of seven possible design changes [2] has led to a selection - of four changes [3] that are proposed to be implemented here. - -Design: - - 1. Shorter Circuit Extension Timeout - - When establishing a connection to a hidden service a client cannibalizes - an existing circuit and extends it by one hop to one of the service's - introduction points. In most cases this can be accomplished within a few - seconds. Therefore, the current timeout of 60 seconds for extending a - circuit is far too high. - - Assuming that the timeout would be reduced to a lower value, for example - 30 seconds, a second (or third) attempt to cannibalize and extend would - be started earlier. With the current timeout of 60 seconds, 93.42% of all - circuits can be established, whereas this fraction would have been only - 0.87% smaller at 92.55% with a timeout of 30 seconds. - - For a timeout of 30 seconds the performance gain would be approximately 2 - seconds in the mean as opposed to the current timeout of 60 seconds. At - the same time a smaller timeout leads to discarding an increasing number - of circuits that might have been completed within the current timeout of - 60 seconds. - - Measurements with simulated low-bandwidth connectivity have shown that - there is no significant effect of client connectivity on circuit - extension times. The reason for this might be that extension messages are - small and thereby independent of the client bandwidth. Further, the - connection between client and entry node only constitutes a single hop of - a circuit, so that its influence on the whole circuit is limited. - - The exact value of the new timeout does not necessarily have to be 30 - seconds, but might also depend on the results of circuit build timeout - measurements as described in proposal 151. - - 2. Parallel Connections to Introduction Points - - An additional approach to accelerate extension of introduction circuits - is to extend a second circuit in parallel to a different introduction - point. Such parallel extension attempts should be started after a short - delay of, e.g., 15 seconds in order to prevent unnecessary circuit - extensions and thereby save network resources. Whichever circuit - extension succeeds first is used for introduction, while the other - attempt is aborted. - - An evaluation has been performed for the more resource-intensive approach - of starting two parallel circuits immediately instead of waiting for a - short delay. The result was a reduction of connection establishment times - from 27.4 seconds in the original protocol to 22.5 seconds. - - While the effect of the proposed approach of delayed parallelization on - mean connection establishment times is expected to be smaller, - variability of connection attempt times can be reduced significantly. - - 3. Increase Count of Internal Circuits - - Hidden services need to create or cannibalize and extend a circuit to a - rendezvous point for every client request. Really popular hidden services - require more than two internal circuits in the pool to answer multiple - client requests at the same time. This scenario was not yet analyzed, but - will probably exhibit worse performance than measured in the previous - analysis. The number of preemptively built internal circuits should be a - function of connection requests in the past to adapt to changing needs. - Furthermore, an increased number of internal circuits on client side - would allow clients to establish connections to more than one hidden - service at a time. - - Under the assumption that a popular hidden service cannot make use of - cannibalization for connecting to rendezvous points, the circuit creation - time needs to be added to the current results. In the mean, the - connection establishment time to a popular hidden service would increase - by 4.7 seconds. - - 4. Build More Introduction Circuits - - When establishing introduction points, a hidden service should launch 5 - instead of 3 introduction circuits at the same time and use only the - first 3 that could be established. The remaining two circuits could still - be used for other purposes afterwards. - - The effect has been simulated using previously measured data, too. - Therefore, circuit establishment times were derived from log files and - written to an array. Afterwards, a simulation with 10,000 runs was - performed picking 5 (4, 6) random values and using the 3 lowest values in - contrast to picking only 3 values at random. The result is that the mean - time of the 3-out-of-3 approach is 8.1 seconds, while the mean time of - the 3-out-of-5 approach is 4.4 seconds. - - The effect on network load is minimal, because the hidden service can - reuse the slower internal circuits for other purposes, e.g., rendezvous - circuits. The only change is that a hidden service starts establishing - more circuits at once instead of subsequently doing so. - -References: - - [1] http://freehaven.net/~karsten/hidserv/perfanalysis-2008-06-15.pdf - - [2] http://freehaven.net/~karsten/hidserv/discussion-2008-07-15.pdf - - [3] http://freehaven.net/~karsten/hidserv/design-2008-08-15.pdf - diff --git a/doc/spec/proposals/156-tracking-blocked-ports.txt b/doc/spec/proposals/156-tracking-blocked-ports.txt deleted file mode 100644 index 1e7b0d963..000000000 --- a/doc/spec/proposals/156-tracking-blocked-ports.txt +++ /dev/null @@ -1,529 +0,0 @@ -Filename: 156-tracking-blocked-ports.txt -Title: Tracking blocked ports on the client side -Version: $Revision$ -Last-Modified: $Date$ -Author: Robert Hogan -Created: 14-Oct-2008 -Status: Open -Target: 0.2.? - -Motivation: -Tor clients that are behind extremely restrictive firewalls can end up -waiting a while for their first successful OR connection to a node on the -network. Worse, the more restrictive their firewall the more susceptible -they are to an attacker guessing their entry nodes. Tor routers that -are behind extremely restrictive firewalls can only offer a limited, -'partitioned' service to other routers and clients on the network. Exit -nodes behind extremely restrictive firewalls may advertise ports that they -are actually not able to connect to, wasting network resources in circuit -constructions that are doomed to fail at the last hop on first use. - -Proposal: - -When a client attempts to connect to an entry guard it should avoid -further attempts on ports that fail once until it has connected to at -least one entry guard successfully. (Maybe it should wait for more than -one failure to reduce the skew on the first node selection.) Thereafter -it should select entry guards regardless of port and warn the user if -it observes that connections to a given port have failed every multiple -of 5 times without success or since the last success. - -Tor should warn the operators of exit, middleman and entry nodes if it -observes that connections to a given port have failed a multiple of 5 -times without success or since the last success. If attempts on a port -fail 20 or more times without or since success, Tor should add the port -to a 'blocked-ports' entry in its descriptor's extra-info. Some thought -needs to be given to what the authorities might do with this information. - -Related TODO item: - "- Automatically determine what ports are reachable and start using - those, if circuits aren't working and it's a pattern we - recognize ("port 443 worked once and port 9001 keeps not - working")." - - -I've had a go at implementing all of this in the attached. - -Addendum: -Just a note on the patch, storing the digest of each router that uses the port -is a bit of a memory hog, and its only real purpose is to provide a count of -routers using that port when warning the user. That could be achieved when -warning the user by iterating through the routerlist instead. - -Index: src/or/connection_or.c -=================================================================== ---- src/or/connection_or.c (revision 17104) -+++ src/or/connection_or.c (working copy) -@@ -502,6 +502,9 @@ - connection_or_connect_failed(or_connection_t *conn, - int reason, const char *msg) - { -+ if ((reason == END_OR_CONN_REASON_NO_ROUTE) || -+ (reason == END_OR_CONN_REASON_REFUSED)) -+ or_port_hist_failure(conn->identity_digest,TO_CONN(conn)->port); - control_event_or_conn_status(conn, OR_CONN_EVENT_FAILED, reason); - if (!authdir_mode_tests_reachability(get_options())) - control_event_bootstrap_problem(msg, reason); -@@ -580,6 +583,7 @@ - /* already marked for close */ - return NULL; - } -+ - return conn; - } - -@@ -909,6 +913,7 @@ - control_event_or_conn_status(conn, OR_CONN_EVENT_CONNECTED, 0); - - if (started_here) { -+ or_port_hist_success(TO_CONN(conn)->port); - rep_hist_note_connect_succeeded(conn->identity_digest, now); - if (entry_guard_register_connect_status(conn->identity_digest, - 1, now) < 0) { -Index: src/or/rephist.c -=================================================================== ---- src/or/rephist.c (revision 17104) -+++ src/or/rephist.c (working copy) -@@ -18,6 +18,7 @@ - static void bw_arrays_init(void); - static void predicted_ports_init(void); - static void hs_usage_init(void); -+static void or_port_hist_init(void); - - /** Total number of bytes currently allocated in fields used by rephist.c. */ - uint64_t rephist_total_alloc=0; -@@ -89,6 +90,25 @@ - digestmap_t *link_history_map; - } or_history_t; - -+/** or_port_hist_t contains our router/client's knowledge of -+ all OR ports offered on the network, and how many servers with each port we -+ have succeeded or failed to connect to. */ -+typedef struct { -+ /** The port this entry is tracking. */ -+ uint16_t or_port; -+ /** Have we ever connected to this port on another OR?. */ -+ unsigned int success:1; -+ /** The ORs using this port. */ -+ digestmap_t *ids; -+ /** The ORs using this port we have failed to connect to. */ -+ digestmap_t *failure_ids; -+ /** Are we excluding ORs with this port during entry selection?*/ -+ unsigned int excluded; -+} or_port_hist_t; -+ -+static unsigned int still_searching = 0; -+static smartlist_t *or_port_hists; -+ - /** When did we last multiply all routers' weighted_run_length and - * total_run_weights by STABILITY_ALPHA? */ - static time_t stability_last_downrated = 0; -@@ -164,6 +184,16 @@ - tor_free(hist); - } - -+/** Helper: free storage held by a single OR port history entry. */ -+static void -+or_port_hist_free(or_port_hist_t *p) -+{ -+ tor_assert(p); -+ digestmap_free(p->ids,NULL); -+ digestmap_free(p->failure_ids,NULL); -+ tor_free(p); -+} -+ - /** Update an or_history_t object hist so that its uptime/downtime - * count is up-to-date as of when. - */ -@@ -1639,7 +1669,7 @@ - tmp_time = smartlist_get(predicted_ports_times, i); - if (*tmp_time + PREDICTED_CIRCS_RELEVANCE_TIME < now) { - tmp_port = smartlist_get(predicted_ports_list, i); -- log_debug(LD_CIRC, "Expiring predicted port %d", *tmp_port); -+ log_debug(LD_HIST, "Expiring predicted port %d", *tmp_port); - smartlist_del(predicted_ports_list, i); - smartlist_del(predicted_ports_times, i); - rephist_total_alloc -= sizeof(uint16_t)+sizeof(time_t); -@@ -1821,6 +1851,12 @@ - tor_free(last_stability_doc); - built_last_stability_doc_at = 0; - predicted_ports_free(); -+ if (or_port_hists) { -+ SMARTLIST_FOREACH(or_port_hists, or_port_hist_t *, p, -+ or_port_hist_free(p)); -+ smartlist_free(or_port_hists); -+ or_port_hists = NULL; -+ } - } - - /****************** hidden service usage statistics ******************/ -@@ -2356,3 +2392,225 @@ - tor_free(fname); - } - -+/** Create a new entry in the port tracking cache for the or_port in -+ * ri. */ -+void -+or_port_hist_new(const routerinfo_t *ri) -+{ -+ or_port_hist_t *result; -+ const char *id=ri->cache_info.identity_digest; -+ -+ if (!or_port_hists) -+ or_port_hist_init(); -+ -+ SMARTLIST_FOREACH(or_port_hists, or_port_hist_t *, tp, -+ { -+ /* Cope with routers that change their advertised OR port or are -+ dropped from the networkstatus. We don't discard the failures of -+ dropped routers because they are still valid when counting -+ consecutive failures on a port.*/ -+ if (digestmap_get(tp->ids, id) && (tp->or_port != ri->or_port)) { -+ digestmap_remove(tp->ids, id); -+ } -+ if (tp->or_port == ri->or_port) { -+ if (!(digestmap_get(tp->ids, id))) -+ digestmap_set(tp->ids, id, (void*)1); -+ return; -+ } -+ }); -+ -+ result = tor_malloc_zero(sizeof(or_port_hist_t)); -+ result->or_port=ri->or_port; -+ result->success=0; -+ result->ids=digestmap_new(); -+ digestmap_set(result->ids, id, (void*)1); -+ result->failure_ids=digestmap_new(); -+ result->excluded=0; -+ smartlist_add(or_port_hists, result); -+} -+ -+/** Create the port tracking cache. */ -+/*XXX: need to call this when we rebuild/update our network status */ -+static void -+or_port_hist_init(void) -+{ -+ routerlist_t *rl = router_get_routerlist(); -+ -+ if (!or_port_hists) -+ or_port_hists=smartlist_create(); -+ -+ if (rl && rl->routers) { -+ SMARTLIST_FOREACH(rl->routers, routerinfo_t *, ri, -+ { -+ or_port_hist_new(ri); -+ }); -+ } -+} -+ -+#define NOT_BLOCKED 0 -+#define FAILURES_OBSERVED 1 -+#define POSSIBLY_BLOCKED 5 -+#define PROBABLY_BLOCKED 10 -+/** Return the list of blocked ports for our router's extra-info.*/ -+char * -+or_port_hist_get_blocked_ports(void) -+{ -+ char blocked_ports[2048]; -+ char *bp; -+ -+ tor_snprintf(blocked_ports,sizeof(blocked_ports),"blocked-ports"); -+ SMARTLIST_FOREACH(or_port_hists, or_port_hist_t *, tp, -+ { -+ if (digestmap_size(tp->failure_ids) >= PROBABLY_BLOCKED) -+ tor_snprintf(blocked_ports+strlen(blocked_ports), -+ sizeof(blocked_ports)," %u,",tp->or_port); -+ }); -+ if (strlen(blocked_ports) == 13) -+ return NULL; -+ bp=tor_strdup(blocked_ports); -+ bp[strlen(bp)-1]='\n'; -+ bp[strlen(bp)]='\0'; -+ return bp; -+} -+ -+/** Revert to client-only mode if we have seen to many failures on a port or -+ * range of ports.*/ -+static void -+or_port_hist_report_block(unsigned int min_severity) -+{ -+ or_options_t *options=get_options(); -+ char failures_observed[2048],possibly_blocked[2048],probably_blocked[2048]; -+ char port[1024]; -+ -+ memset(failures_observed,0,sizeof(failures_observed)); -+ memset(possibly_blocked,0,sizeof(possibly_blocked)); -+ memset(probably_blocked,0,sizeof(probably_blocked)); -+ -+ SMARTLIST_FOREACH(or_port_hists, or_port_hist_t *, tp, -+ { -+ unsigned int failures = digestmap_size(tp->failure_ids); -+ if (failures >= min_severity) { -+ tor_snprintf(port, sizeof(port), " %u (%u failures %s out of %u on the" -+ " network)",tp->or_port,failures, -+ (!tp->success)?"and no successes": "since last success", -+ digestmap_size(tp->ids)); -+ if (failures >= PROBABLY_BLOCKED) { -+ strlcat(probably_blocked, port, sizeof(probably_blocked)); -+ } else if (failures >= POSSIBLY_BLOCKED) -+ strlcat(possibly_blocked, port, sizeof(possibly_blocked)); -+ else if (failures >= FAILURES_OBSERVED) -+ strlcat(failures_observed, port, sizeof(failures_observed)); -+ } -+ }); -+ -+ log_warn(LD_HIST,"%s%s%s%s%s%s%s%s", -+ server_mode(options) && -+ ((min_severity==FAILURES_OBSERVED) || strlen(probably_blocked))? -+ "You should consider disabling your Tor server.":"", -+ (min_severity==FAILURES_OBSERVED)? -+ "Tor appears to be blocked from connecting to a range of ports " -+ "with the result that it cannot connect to one tenth of the Tor " -+ "network. ":"", -+ strlen(failures_observed)? -+ "Tor has observed failures on the following ports: ":"", -+ failures_observed, -+ strlen(possibly_blocked)? -+ "Tor is possibly blocked on the following ports: ":"", -+ possibly_blocked, -+ strlen(probably_blocked)? -+ "Tor is almost certainly blocked on the following ports: ":"", -+ probably_blocked); -+ -+} -+ -+/** Record the success of our connection to digest's -+ * OR port. */ -+void -+or_port_hist_success(uint16_t or_port) -+{ -+ SMARTLIST_FOREACH(or_port_hists, or_port_hist_t *, tp, -+ { -+ if (tp->or_port != or_port) -+ continue; -+ /*Reset our failure stats so we can notice if this port ever gets -+ blocked again.*/ -+ tp->success=1; -+ if (digestmap_size(tp->failure_ids)) { -+ digestmap_free(tp->failure_ids,NULL); -+ tp->failure_ids=digestmap_new(); -+ } -+ if (still_searching) { -+ still_searching=0; -+ SMARTLIST_FOREACH(or_port_hists,or_port_hist_t *,t,t->excluded=0;); -+ } -+ return; -+ }); -+} -+/** Record the failure of our connection to digest's -+ * OR port. Warn, exclude the port from future entry guard selection, or -+ * add port to blocked-ports in our server's extra-info as appropriate. */ -+void -+or_port_hist_failure(const char *digest, uint16_t or_port) -+{ -+ int total_failures=0, ports_excluded=0, report_block=0; -+ int total_routers=smartlist_len(router_get_routerlist()->routers); -+ -+ SMARTLIST_FOREACH(or_port_hists, or_port_hist_t *, tp, -+ { -+ ports_excluded += tp->excluded; -+ total_failures+=digestmap_size(tp->failure_ids); -+ if (tp->or_port != or_port) -+ continue; -+ /* We're only interested in unique failures */ -+ if (digestmap_get(tp->failure_ids, digest)) -+ return; -+ -+ total_failures++; -+ digestmap_set(tp->failure_ids, digest, (void*)1); -+ if (still_searching && !tp->success) { -+ tp->excluded=1; -+ ports_excluded++; -+ } -+ if ((digestmap_size(tp->ids) >= POSSIBLY_BLOCKED) && -+ !(digestmap_size(tp->failure_ids) % POSSIBLY_BLOCKED)) -+ report_block=POSSIBLY_BLOCKED; -+ }); -+ -+ if (total_failures >= (int)(total_routers/10)) -+ or_port_hist_report_block(FAILURES_OBSERVED); -+ else if (report_block) -+ or_port_hist_report_block(report_block); -+ -+ if (ports_excluded >= smartlist_len(or_port_hists)) { -+ log_warn(LD_HIST,"During entry node selection Tor tried every port " -+ "offered on the network on at least one server " -+ "and didn't manage a single " -+ "successful connection. This suggests you are behind an " -+ "extremely restrictive firewall. Tor will keep trying to find " -+ "a reachable entry node."); -+ SMARTLIST_FOREACH(or_port_hists, or_port_hist_t *, tp, tp->excluded=0;); -+ } -+} -+ -+/** Add any ports marked as excluded in or_port_hist_t to rt */ -+void -+or_port_hist_exclude(routerset_t *rt) -+{ -+ SMARTLIST_FOREACH(or_port_hists, or_port_hist_t *, tp, -+ { -+ char portpolicy[9]; -+ if (tp->excluded) { -+ tor_snprintf(portpolicy,sizeof(portpolicy),"*:%u", tp->or_port); -+ log_warn(LD_HIST,"Port %u may be blocked, excluding it temporarily " -+ "from entry guard selection.", tp->or_port); -+ routerset_parse(rt, portpolicy, "Ports"); -+ } -+ }); -+} -+ -+/** Allow the exclusion of ports during our search for an entry node. */ -+void -+or_port_hist_search_again(void) -+{ -+ still_searching=1; -+} -Index: src/or/or.h -=================================================================== ---- src/or/or.h (revision 17104) -+++ src/or/or.h (working copy) -@@ -3864,6 +3864,13 @@ - int any_predicted_circuits(time_t now); - int rep_hist_circbuilding_dormant(time_t now); - -+void or_port_hist_failure(const char *digest, uint16_t or_port); -+void or_port_hist_success(uint16_t or_port); -+void or_port_hist_new(const routerinfo_t *ri); -+void or_port_hist_exclude(routerset_t *rt); -+void or_port_hist_search_again(void); -+char *or_port_hist_get_blocked_ports(void); -+ - /** Possible public/private key operations in Tor: used to keep track of where - * we're spending our time. */ - typedef enum { -Index: src/or/routerparse.c -=================================================================== ---- src/or/routerparse.c (revision 17104) -+++ src/or/routerparse.c (working copy) -@@ -1401,6 +1401,8 @@ - goto err; - } - -+ or_port_hist_new(router); -+ - if (!router->platform) { - router->platform = tor_strdup(""); - } -Index: src/or/router.c -=================================================================== ---- src/or/router.c (revision 17104) -+++ src/or/router.c (working copy) -@@ -1818,6 +1818,7 @@ - char published[ISO_TIME_LEN+1]; - char digest[DIGEST_LEN]; - char *bandwidth_usage; -+ char *blocked_ports; - int result; - size_t len; - -@@ -1825,7 +1826,6 @@ - extrainfo->cache_info.identity_digest, DIGEST_LEN); - format_iso_time(published, extrainfo->cache_info.published_on); - bandwidth_usage = rep_hist_get_bandwidth_lines(1); -- - result = tor_snprintf(s, maxlen, - "extra-info %s %s\n" - "published %s\n%s", -@@ -1835,6 +1835,16 @@ - if (result<0) - return -1; - -+ blocked_ports = or_port_hist_get_blocked_ports(); -+ if (blocked_ports) { -+ result = tor_snprintf(s+strlen(s), maxlen-strlen(s), -+ "%s", -+ blocked_ports); -+ tor_free(blocked_ports); -+ if (result<0) -+ return -1; -+ } -+ - if (should_record_bridge_info(options)) { - static time_t last_purged_at = 0; - char *geoip_summary; -Index: src/or/circuitbuild.c -=================================================================== ---- src/or/circuitbuild.c (revision 17104) -+++ src/or/circuitbuild.c (working copy) -@@ -62,6 +62,7 @@ - - static void entry_guards_changed(void); - static time_t start_of_month(time_t when); -+static int num_live_entry_guards(void); - - /** Iterate over values of circ_id, starting from conn-\>next_circ_id, - * and with the high bit specified by conn-\>circ_id_type, until we get -@@ -1627,12 +1628,14 @@ - smartlist_t *excluded; - or_options_t *options = get_options(); - router_crn_flags_t flags = 0; -+ routerset_t *_ExcludeNodes; - - if (state && options->UseEntryGuards && - (purpose != CIRCUIT_PURPOSE_TESTING || options->BridgeRelay)) { - return choose_random_entry(state); - } - -+ _ExcludeNodes = routerset_new(); - excluded = smartlist_create(); - - if (state && (r = build_state_get_exit_router(state))) { -@@ -1670,12 +1673,18 @@ - if (options->_AllowInvalid & ALLOW_INVALID_ENTRY) - flags |= CRN_ALLOW_INVALID; - -+ if (options->ExcludeNodes) -+ routerset_union(_ExcludeNodes,options->ExcludeNodes); -+ -+ or_port_hist_exclude(_ExcludeNodes); -+ - choice = router_choose_random_node( - NULL, - excluded, -- options->ExcludeNodes, -+ _ExcludeNodes, - flags); - smartlist_free(excluded); -+ routerset_free(_ExcludeNodes); - return choice; - } - -@@ -2727,6 +2736,7 @@ - entry_guards_update_state(or_state_t *state) - { - config_line_t **next, *line; -+ unsigned int have_reachable_entry=0; - if (! entry_guards_dirty) - return; - -@@ -2740,6 +2750,7 @@ - char dbuf[HEX_DIGEST_LEN+1]; - if (!e->made_contact) - continue; /* don't write this one to disk */ -+ have_reachable_entry=1; - *next = line = tor_malloc_zero(sizeof(config_line_t)); - line->key = tor_strdup("EntryGuard"); - line->value = tor_malloc(HEX_DIGEST_LEN+MAX_NICKNAME_LEN+2); -@@ -2785,6 +2796,11 @@ - if (!get_options()->AvoidDiskWrites) - or_state_mark_dirty(get_or_state(), 0); - entry_guards_dirty = 0; -+ -+ /* XXX: Is this the place to decide that we no longer have any reachable -+ guards? */ -+ if (!have_reachable_entry) -+ or_port_hist_search_again(); - } - - /** If question is the string "entry-guards", then dump - diff --git a/doc/spec/proposals/157-specific-cert-download.txt b/doc/spec/proposals/157-specific-cert-download.txt deleted file mode 100644 index e54a98727..000000000 --- a/doc/spec/proposals/157-specific-cert-download.txt +++ /dev/null @@ -1,104 +0,0 @@ -Filename: 157-specific-cert-download.txt -Title: Make certificate downloads specific -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 2-Dec-2008 -Status: Accepted -Target: 0.2.1.x - -History: - - 2008 Dec 2, 22:34 - Changed name of cross certification field to match the other authority - certificate fields. - -Status: - - As of 0.2.1.9-alpha: - Cross-certification is implemented for new certificates, but not yet - required. Directories support the tor/keys/fp-sk urls. - -Overview: - - Tor's directory specification gives two ways to download a certificate: - by its identity fingerprint, or by the digest of its signing key. Both - are error-prone. We propose a new download mechanism to make sure that - clients get the certificates they want. - -Motivation: - - When a client wants a certificate to verify a consensus, it has two choices - currently: - - Download by identity key fingerprint. In this case, the client risks - getting a certificate for the same authority, but with a different - signing key than the one used to sign the consensus. - - - Download by signing key fingerprint. In this case, the client risks - getting a forged certificate that contains the right signing key - signed with the wrong identity key. (Since caches are willing to - cache certs from authorities they do not themselves recognize, the - attacker wouldn't need to compromise an authority's key to do this.) - -Current solution: - - Clients fetch by identity keys, and re-fetch with backoff if they don't get - certs with the signing key they want. - -Proposed solution: - - Phase 1: Add a URL type for clients to download certs by identity _and_ - signing key fingerprint. Unless both fields match, the client doesn't - accept the certificate(s). Clients begin using this method when their - randomly chosen directory cache supports it. - - Phase 1A: Simultaneously, add a cross-certification element to - certificates. - - Phase 2: Once many directory caches support phase 1, clients should prefer - to fetch certificates using that protocol when available. - - Phase 2A: Once all authorities are generating cross-certified certificates - as in phase 1A, require cross-certification. - -Specification additions: - - The key certificate whose identity key fingerprint is and whose signing - key fingerprint is should be available at: - - http:///tor/keys/fp-sk/-.z - - As usual, clients may request multiple certificates using: - - http:///tor/keys/fp-sk/-+-.z - - Clients SHOULD use this format whenever they know both key fingerprints for - a desired certificate. - - - Certificates SHOULD contain the following field (at most once): - - "dir-key-crosscert" NL CrossSignature NL - - where CrossSignature is a signature, made using the certificate's signing - key, of the digest of the PKCS1-padded hash of the certificate's identity - key. For backward compatibility with broken versions of the parser, we - wrap the base64-encoded signature in -----BEGIN ID SIGNATURE---- and - -----END ID SIGNATURE----- tags. (See bug 880.) Implementations MUST allow - the "ID " portion to be omitted, however. - - When encountering a certificate with a dir-key-crosscert entry, - implementations MUST verify that the signature is a correct signature of - the hash of the identity key using the signing key. - - (In a future version of this specification, dir-key-crosscert entries will - be required.) - -Why cross-certify too? - - Cross-certification protects clients who haven't updated yet, by reducing - the number of caches that are willing to hold and serve bogus certificates. - -References: - - This is related to part 2 of bug 854. diff --git a/doc/spec/proposals/158-microdescriptors.txt b/doc/spec/proposals/158-microdescriptors.txt deleted file mode 100644 index f478a3c83..000000000 --- a/doc/spec/proposals/158-microdescriptors.txt +++ /dev/null @@ -1,207 +0,0 @@ -Filename: 158-microdescriptors.txt -Title: Clients download consensus + microdescriptors -Version: $Revision$ -Last-Modified: $Date$ -Author: Roger Dingledine -Created: 17-Jan-2009 -Status: Open - -1. Overview - - This proposal replaces section 3.2 of proposal 141, which was - called "Fetching descriptors on demand". Rather than modifying the - circuit-building protocol to fetch a server descriptor inline at each - circuit extend, we instead put all of the information that clients need - either into the consensus itself, or into a new set of data about each - relay called a microdescriptor. The microdescriptor is a direct - transform from the relay descriptor, so relays don't even need to know - this is happening. - - Descriptor elements that are small and frequently changing should go - in the consensus itself, and descriptor elements that are small and - relatively static should go in the microdescriptor. If we ever end up - with descriptor elements that aren't small yet clients need to know - them, we'll need to resume considering some design like the one in - proposal 141. - -2. Motivation - - See - http://archives.seul.org/or/dev/Nov-2008/msg00000.html and - http://archives.seul.org/or/dev/Nov-2008/msg00001.html and especially - http://archives.seul.org/or/dev/Nov-2008/msg00007.html - for a discussion of the options and why this is currently the best - approach. - -3. Design - - There are three pieces to the proposal. First, authorities will list in - their votes (and thus in the consensus) what relay descriptor elements - are included in the microdescriptor, and also list the expected hash - of microdescriptor for each relay. Second, directory mirrors will serve - microdescriptors. Third, clients will ask for them and cache them. - -3.1. Consensus changes - - V3 votes should include a new line: - microdescriptor-elements bar baz foo - listing each descriptor element (sorted alphabetically) that authority - included when it calculated its expected microdescriptor hashes. - - We also need to include the hash of each expected microdescriptor in - the routerstatus section. I suggest a new "m" line for each stanza, - with the base64 of the hash of the elements that the authority voted - for above. - - The consensus microdescriptor-elements and "m" lines are then computed - as described in Section 3.1.2 below. - - I believe that means we need a new consensus-method "6" that knows - how to compute the microdescriptor-elements and add "m" lines. - -3.1.1. Descriptor elements to include for now - - To start, the element list that authorities suggest should be - family onion-key - - (Note that the or-dev posts above only mention onion-key, but if - we don't also include family then clients will never learn it. It - seemed like it should be relatively static, so putting it in the - microdescriptor is smarter than trying to fit it into the consensus.) - - We could imagine a config option "family,onion-key" so authorities - could change their voted preferences without needing to upgrade. - -3.1.2. Computing consensus for microdescriptor-elements and "m" lines - - One approach is for the consensus microdescriptor-elements line to - include every element listed by a majority of authorities, sorted. The - problem here is that it will no longer be deterministic what the correct - hash for the "m" line should be. We could imagine telling the authority - to go look in its descriptor and produce the right hash itself, but - we don't want consensus calculation to be based on external data like - that. (Plus, the authority may not have the descriptor that everybody - else voted to use.) - - The better approach is to take the exact set that has the most votes - (breaking ties by the set that has the most elements, and breaking - ties after that by whichever is alphabetically first). That will - increase the odds that we actually get a microdescriptor hash that - is both a) for the descriptor we're putting in the consensus, and b) - over the elements that we're declaring it should be for. - - Then the "m" line for a given relay is the one that gets the most votes - from authorities that both a) voted for the microdescriptor-elements - line we're using, and b) voted for the descriptor we're using. - - (If there's a tie, use the smaller hash. But really, if there are - multiple such votes and they differ about a microdescriptor, we caught - one of them lying or being buggy. We should log it to track down why.) - - If there are no such votes, then we leave out the "m" line for that - relay. That means clients should avoid it for this time period. (As - an extension it could instead mean that clients should fetch the - descriptor and figure out its microdescriptor themselves. But let's - not get ahead of ourselves.) - - It would be nice to have a more foolproof way to agree on what - microdescriptor hash each authority should vote for, so we can avoid - missing "m" lines. Just switching to a new consensus-method each time - we change the set of microdescriptor-elements won't help though, since - each authority will still have to decide what hash to vote for before - knowing what consensus-method will be used. - - Here's one way we could do it. Each vote / consensus includes - the microdescriptor-elements that were used to compute the hashes, - and also a preferred-microdescriptor-elements set. If an authority - has a consensus from the previous period, then it should use the - consensus preferred-microdescriptor-elements when computing its votes - for microdescriptor-elements and the appropriate hashes in the upcoming - period. (If it has no previous consensus, then it just writes its - own preferences in both lines.) - -3.2. Directory mirrors serve microdescriptors - - Directory mirrors should then read the microdescriptor-elements line - from the consensus, and learn how to answer requests. (Directory mirrors - continue to serve normal relay descriptors too, a) to serve old clients - and b) to be able to construct microdescriptors on the fly.) - - The microdescriptors with hashes ,, should be available at: - http:///tor/micro/d/++.z - - All the microdescriptors from the current consensus should also be - available at: - http:///tor/micro/all.z - so a client that's bootstrapping doesn't need to send a 70KB URL just - to name every microdescriptor it's looking for. - - The format of a microdescriptor is the header line - "microdescriptor-header" - followed by each element (keyword and body), alphabetically. There's - no need to mention what hash it's for, since it's self-identifying: - you can hash the elements to learn this. - - (Do we need a footer line to show that it's over, or is the next - microdescriptor line or EOF enough of a hint? A footer line wouldn't - hurt much. Also, no fair voting for the microdescriptor-element - "microdescriptor-header".) - - The hash of the microdescriptor is simply the hash of the concatenated - elements -- not counting the header line or hypothetical footer line. - Unless you prefer that? - - Is there a reasonable way to version these things? We could say that - the microdescriptor-header line can contain arguments which clients - must ignore if they don't understand them. Any better ways? - - Directory mirrors should check to make sure that the microdescriptors - they're about to serve match the right hashes (either the hashes from - the fetch URL or the hashes from the consensus, respectively). - - We will probably want to consider some sort of smart data structure to - be able to quickly convert microdescriptor hashes into the appropriate - microdescriptor. Clients will want this anyway when they load their - microdescriptor cache and want to match it up with the consensus to - see what's missing. - -3.3. Clients fetch them and cache them - - When a client gets a new consensus, it looks to see if there are any - microdescriptors it needs to learn. If it needs to learn more than - some threshold of the microdescriptors (half?), it requests 'all', - else it requests only the missing ones. - - Clients maintain a cache of microdescriptors along with metadata like - when it was last referenced by a consensus. They keep a microdescriptor - until it hasn't been mentioned in any consensus for a week. Future - clients might cache them for longer or shorter times. - -3.3.1. Information leaks from clients - - If a client asks you for a set of microdescs, then you know she didn't - have them cached before. How much does that leak? What about when - we're all using our entry guards as directory guards, and we've seen - that user make a bunch of circuits already? - - Fetching "all" when you need at least half is a good first order fix, - but might not be all there is to it. - - Another future option would be to fetch some of the microdescriptors - anonymously (via a Tor circuit). - -4. Transition and deployment - - Phase one, the directory authorities should start voting on - microdescriptors and microdescriptor elements, and putting them in the - consensus. This should happen during the 0.2.1.x series, and should - be relatively easy to do. - - Phase two, directory mirrors should learn how to serve them, and learn - how to read the consensus to find out what they should be serving. This - phase could be done either in 0.2.1.x or early in 0.2.2.x, depending - on how messy it turns out to be and how quickly we get around to it. - - Phase three, clients should start fetching and caching them instead - of normal descriptors. This should happen post 0.2.1.x. - diff --git a/doc/spec/proposals/159-exit-scanning.txt b/doc/spec/proposals/159-exit-scanning.txt deleted file mode 100644 index fbc69aa9e..000000000 --- a/doc/spec/proposals/159-exit-scanning.txt +++ /dev/null @@ -1,144 +0,0 @@ -Filename: 159-exit-scanning.txt -Title: Exit Scanning -Version: $Revision$ -Last-Modified: $Date$ -Author: Mike Perry -Created: 13-Feb-2009 -Status: Open - -Overview: - -This proposal describes the implementation and integration of an -automated exit node scanner for scanning the Tor network for malicious, -misconfigured, firewalled or filtered nodes. - -Motivation: - -Tor exit nodes can be run by anyone with an Internet connection. Often, -these users aren't fully aware of limitations of their networking -setup. Content filters, antivirus software, advertisements injected by -their service providers, malicious upstream providers, and the resource -limitations of their computer or networking equipment have all been -observed on the current Tor network. - -It is also possible that some nodes exist purely for malicious -purposes. In the past, there have been intermittent instances of -nodes spoofing SSH keys, as well as nodes being used for purposes of -plaintext surveillance. - -While it is not realistic to expect to catch extremely targeted or -completely passive malicious adversaries, the goal is to prevent -malicious adversaries from deploying dragnet attacks against large -segments of the Tor userbase. - - -Scanning methodology: - -The first scans to be implemented are HTTP, HTML, Javascript, and -SSL scans. - -The HTTP scan scrapes Google for common filetype urls such as exe, msi, -doc, dmg, etc. It then fetches these urls through Non-Tor and Tor, and -compares the SHA1 hashes of the resulting content. - -The SSL scan downloads certificates for all IPs a domain will locally -resolve to and compares these certificates to those seen over Tor. The -scanner notes if a domain had rotated certificates locally in the -results for each scan. - -The HTML scan checks HTML, Javascript, and plugin content for -modifications. Because of the dynamic nature of most of the web, the -scanner has a number of mechanisms built in to filter out false -positives that are used when a change is noticed between Tor and -Non-Tor. - -All tests also share a URL-based false positive filter that -automatically removes results retroactively if the number of failures -exceeds a certain percentage of nodes tested with the URL. - - -Deployment Stages: - -To avoid instances where bugs cause us to mark exit nodes as BadExit -improperly, it is proposed that we begin use of the scanner in stages. - -1. Manual Review: - - In the first stage, basic scans will be run by a small number of - people while we stabilize the scanner. The scanner has the ability - to resume crashed scans, and to rescan nodes that fail various - tests. - -2. Human Review: - - In the second stage, results will be automatically mailed to - an email list of interested parties for review. We will also begin - classifying failure types into three to four different severity - levels, based on both the reliability of the test and the nature of - the failure. - -3. Automatic BadExit Marking: - - In the final stage, the scanner will begin marking exits depending - on the failure severity level in one of three different ways: by - node idhex, by node IP, or by node IP mask. A potential fourth, less - severe category of results may still be delivered via email only for - review. - - BadExit markings will be delivered in batches upon completion - of whole-network scans, so that the final false positive - filter has an opportunity to filter out URLs that exhibit - dynamic content beyond what we can filter. - - -Specification of Exit Marking: - -Technically, BadExit could be marked via SETCONF AuthDirBadExit over -the control port, but this would allow full access to the directory -authority configuration and operation. - -The approved-routers file could also be used, but currently it only -supports fingerprints, and it also contains other data unrelated to -exit scanning that would be difficult to coordinate. - -Instead, we propose that a new badexit-routers file that has three -keywords: - - BadExitNet 1*[exitpattern from 2.3 in dir-spec.txt] - BadExitFP 1*[hexdigest from 2.3 in dir-spec.txt] - -BadExitNet lines would follow the codepaths used by AuthDirBadExit to -set authdir_badexit_policy, and BadExitFP would follow the codepaths -from approved-router's !badexit lines. - -The scanner would have exclusive ability to write, append, rewrite, -and modify this file. Prior to building a new consensus vote, a -participating Tor authority would read in a fresh copy. - - -Security Implications: - -Aside from evading the scanner's detection, there are two additional -high-level security considerations: - -1. Ensure nodes cannot be marked BadExit by an adversary at will - -It is possible individual website owners will be able to target certain -Tor nodes, but once they begin to attempt to fail more than the URL -filter percentage of the exits, their sites will be automatically -discarded. - -Failing specific nodes is possible, but scanned results are fully -reproducible, and BadExits should be rare enough that humans are never -fully removed from the loop. - -State (cookies, cache, etc) does not otherwise persist in the scanner -between exit nodes to enable one exit node to bias the results of a -later one. - -2. Ensure that scanner compromise does not yield authority compromise - -Having a separate file that is under the exclusive control of the -scanner allows us to heavily isolate the scanner from the Tor -authority, potentially even running them on separate machines. - diff --git a/doc/spec/proposals/ideas/xxx-auto-update.txt b/doc/spec/proposals/ideas/xxx-auto-update.txt deleted file mode 100644 index dc9a857c1..000000000 --- a/doc/spec/proposals/ideas/xxx-auto-update.txt +++ /dev/null @@ -1,39 +0,0 @@ - -Notes on an auto updater: - -steve wants a "latest" symlink so he can always just fetch that. - -roger worries that this will exacerbate the "what version are you -using?" "latest." problem. - -weasel suggests putting the latest recommended version in dns. then -we don't have to hit the website. it's got caching, it's lightweight, -it scales. just put it in a TXT record or something. - -but, no dnssec. - -roger suggests a file on the https website that lists the latest -recommended version (or filename or url or something like that). - -(steve seems to already be doing this with xerobank. he additionally -suggests a little blurb that can be displayed to the user to describe -what's new.) - -how to verify you're getting the right file? -a) it's https. -b) ship with a signing key, and use some openssl functions to verify. -c) both - -andrew reminds us that we have a "recommended versions" line in the -consensus directory already. - -if only we had some way to point out the "latest stable recommendation" -from this list. we could list it first, or something. - -the recommended versions line also doesn't take into account which -packages are available -- e.g. on Windows one version might be the best -available, and on OS X it might be a different one. - -aren't there existing solutions to this? surely there is a beautiful, -efficient, crypto-correct auto updater lib out there. even for windows. - diff --git a/doc/spec/proposals/ideas/xxx-bridge-disbursement.txt b/doc/spec/proposals/ideas/xxx-bridge-disbursement.txt deleted file mode 100644 index 6c9a3c71e..000000000 --- a/doc/spec/proposals/ideas/xxx-bridge-disbursement.txt +++ /dev/null @@ -1,174 +0,0 @@ - -How to hand out bridges. - -Divide bridges into 'strategies' as they come in. Do this uniformly -at random for now. - -For each strategy, we'll hand out bridges in a different way to -clients. This document describes two strategies: email-based and -IP-based. - -0. Notation: - - HMAC(k,v) : an HMAC of v using the key k. - - A|B: The string A concatenated with the string B. - - -1. Email-based. - - Goal: bootstrap based on one or more popular email service's sybil - prevention algorithms. - - - Parameters: - HMAC -- an HMAC function - P -- a time period - K -- the number of bridges to send in a period. - - Setup: Generate two nonces, N and M. - - As bridges arrive, put them into a ring according to HMAC(N,ID) - where ID is the bridges's identity digest. - - Divide time into divisions of length P. - - When we get an email: - - If it's not from a supported email service, reject it. - - If we already sent a response to that email address (normalized) - in this period, send _exactly_ the same response. - - If it is from a supported service, generate X = HMAC(M,PS|E) where E - is the lowercased normalized email address for the user, and - where PS is the start of the currrent period. Send - the first K bridges in the ring after point X. - - [If we want to make sure that repeat queries are given exactly the - same results, then we can't let the ring change during the - time period. For a long time period like a month, that's quite a - hassle. How about instead just keeping a replay cache of addresses - that have been answered, and sending them a "sorry, you already got - your addresses for the time period; perhaps you should try these - other fine distribution strategies while you wait?" response? This - approach would also resolve the "Make sure you can't construct a - distinct address to match an existing one" note below. -RD] - - [I think, if we get a replay, we need to send back the same - answer as we did the first time, not say "try again." - Otherwise we need to worry that an attacker can keep people - from getting bridges by preemtively asking for them, - or that an attacker may force them to prove they haven't - gotten any bridges by asking. -NM] - - [While we're at it, if we do the replay cache thing and don't need - repeatable answers, we could just pick K random answers from the - pool. Is it beneficial that a bridge user who knows about a clump of - nodes will be sharing them with other users who know about a similar - (overlapping) clump? One good aspect is against an adversary who - learns about a clump this way and watches those bridges to learn - other users and discover *their* bridges: he doesn't learn about - as many new bridges as he might if they were randomly distributed. - A drawback is against an adversary who happens to pick two email - addresses in P that include overlapping answers: he can measure - the difference in clumps and estimate how quickly the bridge pool - is growing. -RD] - - [Random is one more darn thing to implement; rings are already - there. -NM] - - [If we make the period P be mailbox-specific, and make it a random - value around some mean, then we make it harder for an attacker to - know when to try using his small army of gmail addresses to gather - another harvest. But we also make it harder for users to know when - they can try again. -RD] - - [Letting the users know about when they can try again seems - worthwhile. Otherwise users and attackers will all probe and - probe and probe until they get an answer. No additional - security will be achieved, but bandwidth will be lost. -NM] - - To normalize an email address: - Start with the RFC822 address. Consider only the mailbox {???} - portion of the address (username@domain). Put this into lowercase - ascii. - - Questions: - What to do with weird character encodings? Look up the RFC. - - Notes: - Make sure that you can't force a single email address to appear - in lots of different ways. IOW, if nickm@freehaven.net and - NICKM@freehaven.net aren't treated the same, then I can get lots - more bridges than I should. - - Make sure you can't construct a distinct address to match an - existing one. IOW, if we treat nickm@X and nickm@Y as the same - user, then anybody can register nickm@Z and use it to tell which - bridges nickm@X got (or would get). - - Make sure that we actually check headers so we can't be trivially - used to spam people. - - -2. IP-based. - - Goal: avoid handing out all the bridges to users in a similar IP - space and time. - - Parameters: - - T_Flush -- how long it should take a user on a single network to - see a whole cluster of bridges. - - N_C - - K -- the number of bridges we hand out in response to a single - request. - - Setup: using an AS map or a geoip map or some other flawed input - source, divide IP space into "areas" such that surveying a large - collection of "areas" is hard. For v0, use /24 address blocks. - - Group areas into N_C clusters. - - Generate secrets L, M, N. - - Set the period P such that P*(bridges-per-cluster/K) = T_flush. - Don't set P to greater than a week, or less than three hours. - - When we get a bridge: - - Based on HMAC(L,ID), assign the bridge to a cluster. Within each - cluster, keep the bridges in a ring based on HMAC(M,ID). - - [Should we re-sort the rings for each new time period, so the ring - for a given cluster is based on HMAC(M,PS|ID)? -RD] - - When we get a connection: - - If it's http, redirect it to https. - - Let area be the incoming IP network. Let PS be the current - period. Compute X = HMAC(N, PS|area). Return the next K bridges - in the ring after X. - - [Don't we want to compute C = HMAC(key, area) to learn what cluster - to answer from, and then X = HMAC(key, PS|area) to pick a point in - that ring? -RD] - - - Need to clarify that some HMACs are for rings, and some are for - partitions. How rings scale is clear. How do we grow the number of - partitions? Looking at successive bits from the HMAC output is one way. - -3. Open issues - - Denial of service attacks - A good view of network topology - -at some point we should learn some reliability stats on our bridges. when -we say above 'give out k bridges', we might give out 2 reliable ones and -k-2 others. we count around the ring the same way we do now, to find them. - diff --git a/doc/spec/proposals/ideas/xxx-controllers-intercept-extends.txt b/doc/spec/proposals/ideas/xxx-controllers-intercept-extends.txt deleted file mode 100644 index 76ba5c84b..000000000 --- a/doc/spec/proposals/ideas/xxx-controllers-intercept-extends.txt +++ /dev/null @@ -1,44 +0,0 @@ -Author: Geoff Goodell -Title: Allow controller to manage circuit extensions -Date: 12 March 2006 - -History: - - This was once bug 268. Moving it into the proposal system for posterity. - -Test: - -Tor controllers should have a means of learning more about circuits built -through Tor routers. Specifically, if a Tor controller is connected to a Tor -router, it should be able to subscribe to a new class of events, perhaps -"onion" or "router" events. A Tor router SHOULD then ensure that the -controller is informed: - -(a) (NEW) when it receives a connection from some other location, in which -case it SHOULD indicate (1) a unique identifier for the circuit, and (2) a -ServerID in the event of an OR connection from another Tor router, and -Hostname otherwise. - -(b) (REQUEST) when it receives a request to extend an existing circuit to a -successive Tor router, in which case it SHOULD provide (1) the unique -identifier for the circuit, (2) a Hostname (or, if possible, ServerID) of the -previous Tor router in the circuit, and (3) a ServerID for the requested -successive Tor router in the circuit; - -(c) (EXTEND) Tor will attempt to extend the circuit to some other router, in -which case it SHOULD provide the same fields as provided for REQUEST. - -(d) (SUCCEEDED) The circuit has been successfully extended to some ther -router, in which case it SHOULD provide the same fields as provided for -REQUEST. - -We also need a new configuration option analogous to _leavestreamsunattached, -specifying whether the controller is to manage circuit extensions or not. -Perhaps we can call it "_leavecircuitsunextended". When set to 0, Tor -manages everything as usual. When set to 1, a circuit received by the Tor -router cannot transition from "REQUEST" to "EXTEND" state without being -directed by a new controller command. The controller command probably does -not need any arguments, since circuits are extended per client source -routing, and all that the controller does is accept or reject the extension. - -This feature can be used as a basis for enforcing routing policy. diff --git a/doc/spec/proposals/ideas/xxx-exit-scanning-outline.txt b/doc/spec/proposals/ideas/xxx-exit-scanning-outline.txt deleted file mode 100644 index d84094400..000000000 --- a/doc/spec/proposals/ideas/xxx-exit-scanning-outline.txt +++ /dev/null @@ -1,44 +0,0 @@ -1. Scanning process - A. Non-HTML/JS HTTP mime types compared via SHA1 hash - B. Dynamic HTTP content filtered at 4 levels: - 1. IP change+Tor cookie utilization - - Tor cookies replayed with new IP in case of changes - 2. HTML Tag+Attribute+JS comparison - - Comparisons made based only on "relevant" HTML tags - and attributes - 3. HTML Tag+Attribute+JS diffing - - Tags, attributes and JS AST nodes that change during - Non-Tor fetches pruned from comparison - 4. URLS with > N% of node failures removed - - results purged from filesystem at end of scan loop - C. SSL scanning handles some forms of dynamic certs - 1. Catalogs certs for all IPs resolved locally - by getaddrinfo over the duration of the scan. - - Updated each test. - 2. If the domain presents a new cert for each IP, this - is noted on the failure result for the node - 3. If the same IP presents two different certs locally, - the cert list is first refreshed, and if it happens - again, discarded - 4. A N% node failure filter also applies - D. Scanner can be restarted from any point in the event - of scanner or system crashes, or graceful shutdown. - - Results+scan state pickled to filesystem continuously -2. Cron job checks results periodically for reporting - A. Divide failures into three types of BadExit based on type - and frequency over time and incident rate - B. write reject lines to approved-routers for those three types: - 1. ID Hex based (for misconfig/network problems easily fixed) - 2. IP based (for content modification) - 3. IP+mask based (for continuous/egregious content modification) - C. Emails results to tor-scanners@freehaven.net -3. Human Review and Appeal - A. ID Hex-based BadExit is meant to be possible to removed easily - without needing to beg us. - - Should this behavior be encouraged? - B. Optionally can reserve IP based badexits for human review - 1. Results are encapsulated fully on the filesystem and can be - reviewed without network access - 2. Soat has --rescan to rescan failed nodes from a data directory - - New set of URLs used - diff --git a/doc/spec/proposals/ideas/xxx-geoip-survey-plan.txt b/doc/spec/proposals/ideas/xxx-geoip-survey-plan.txt deleted file mode 100644 index 49c6615a6..000000000 --- a/doc/spec/proposals/ideas/xxx-geoip-survey-plan.txt +++ /dev/null @@ -1,137 +0,0 @@ - - -Abstract - - This document explains how to tell about how many Tor users there - are, and how many there are in which country. Statistics are - involved. - -Motivation - - There are a few reasons we need to keep track of which countries - Tor users (in aggregate) are coming from: - - - Resource allocation. Knowing about underserved countries with - lots of users can let us know about where we need to direct - translation and outreach efforts. - - - Anticensorship. Sudden drops in usage on a national basis can - indicate the arrival of a censorious firewall. - - - Sponsor outreach and self-evalutation. Many people and - organizations who are interested in funding The Tor Project's - work want to know that we're successfully serving parts of the - world they're interested in, and that efforts to expand our - userbase are actually succeeding. So do we. - -Goals - - We want to know approximately how many Tor users there are, and which - countries they're in, even in the presence of a hypothetical - "directory guard" feature. Some uncertainty is okay, but we'd like - to be able to put a bound on the uncertainty. - - We need to make sure this information isn't exposed in a way that - helps an adversary. - -Methods for current clients: - - Every client downloads network status documents. There are - currently three methods (one hypothetical) for clients to get them. - - 0.1.2.x clients (and earlier) fetch a v2 networkstatus - document about every NETWORKSTATUS_CLIENT_DL_INTERVAL [30 - minutes]. - - - 0.2.0.x clients fetch a v3 networkstatus consensus document - at a random interval between when their current document is no - longer freshest, and when their current document is about to - expire. - - [In both of the above cases, clients choose a running - directory cache at random with odds roughly proportional to - its bandwidth. If they're just starting, they know a XXXX FIXME -NM] - - - In some future version, clients will choose directory caches - to serve as their "directory guards" to avoid profiling - attacks, similarly to how clients currently start all their - circuits at guard nodes. - - We assume that a directory cache can tell which of these three - categories a client is in by the format of its status request. - - A directory cache can be made to count distinct client IP - addresses that make a certain request of it in a given timeframe, - and total requests made to it over that timeframe. For the first - two cases, a cache can get a picture of the overall - number and countries of users in the network by dividing the IP - count by the probability with which they (as a cache) would be - chosen. Assuming that our listed bandwidth is such that we expect - to be chosen with probability P for any given request, and we've - been counting IPs for long enough that we expect the average - client to have made N requests, they will have visited us at least - once with probability P' = 1-(1-P)^N, and so we divide the IP - counts we've seen by P' for our estimate. To estimate total - number of clients of a given type, determine how many requests a - client of that type will make over that time, and assume we'll - have seen P of them. - - Both of these numbers are useful: the IP counts will give the - total number of IPs connecting to the network, and the request - counts will give the total number of users on the network at any - given time. - - Notes: - - [Over H hours, the N for V2 clients is 2*H, and the N for V3 - clients is currently around H/2 or H/3.] - - - (We should only count requests that we actually intend to answer; - 503 requests shouldn't count.) - - - These measurements should also be taken at a directory - authority if possible: their picture of the network is skewed - by clients that fetch from them directly. These clients, - however, are all the clients that are just bootstrapping - (assuming that the fallback-consensus feature isn't yet used - much). - - - These measurements also overestimate the V2 download rate if - some downloads fail and clients retry them later after backing - off. - -Methods for directory guards: - - If directory guards are in use, directory guards get a picture of - all those users who chose them as a guard when they were listed - as a good choice for a guard, and who are also on the network - now. The cleanest data here will come from nodes that were listed - as good new-guards choices for a while, and have not been so for a - while longer (to study decay rates); nodes that have been listed - as good new-guard choices consistently for a long time (to get a - sample of the network); and nodes that have been listed as good - new-guard choices only recently (to get a sample of new users and - users whose guards have died out.) - - Since directory guards are currently unspecified, we'll need to - make some guesses about how they'll turn out to work. Here are - a couple of approaches that could work. - - We could have clients pick completely new directory guards on - a rolling basis every two months or so. This would ensure - that staying as a guard for a while would be sufficient to - see a sample of users. This is potentially advantageous for - load-balancing the network as well, though it might lose some - of the benefits of directory guard. We need to quantify the - impact of this; it might not actually make stuff worse in - practice, if most guards don't stay good guards for a month - or two. - - - We could try to collect statistics at several directory - guards and combine their statisics, but we would need to make - sure that for all time, at least one of the directory guards - had been recommended as a good choice for new guards. By - looking at new-IP rates for guards, we could get an idea of - user uptake; for looking at old-IP decay rates, we could get - an idea of turnover. This approach would entail significant - complexity, and we'd probably need to record more information - than we'd really like to. - - diff --git a/doc/spec/proposals/ideas/xxx-grand-scaling-plan.txt b/doc/spec/proposals/ideas/xxx-grand-scaling-plan.txt deleted file mode 100644 index 336798cc0..000000000 --- a/doc/spec/proposals/ideas/xxx-grand-scaling-plan.txt +++ /dev/null @@ -1,97 +0,0 @@ - -Right now as I understand it, there are n big scaling problems heading -our way: - -1) Clients need to learn all the relay descriptors they could use. That's -a lot of bytes through a potentially small pipe. -2) Relays need to hold open TCP connections to most other relays. -3) Clients need to learn the whole networkstatus. Even using v3, as -the network grows that will become unwieldy. -4) Dir mirrors need to mirror all the relay descriptors; eventually this -will get big too. - -Here's my plan. - --------------------------------------------------------------------- - -Piece one: download O(1) descriptors rather than O(n) descriptors. - -We need to change our circuit extend protocol so it fetches a relay -descriptor at every 'extend' operation: - - Client fetches networkstatus, picks guards, connects to one. - - Client picks middle hop out of networkstatus, asks guard for - its descriptor, then extends to it. - - Clients picks exit hop out of networkstatus, asks middle hop - for its descriptor, then extends to it. Done. - -The client needs to ask for the descriptor even if it already has a -copy, because otherwise we leak too much. Also, the descriptor needs to -be padded to some large (but not too large) size to prevent the middle -hops from guessing about it. - -The first step towards this is to instrument the current code to see -how much of a win this would actually be -- I am guessing it is already -a win even with the current number of descriptors. - -We also would need to assign the 'Exit' flag more usefully, and make -clients pay attention to it when picking their last hop, since they -don't actually know the exit policies of the relays they're choosing from. - -We also need to think harder about other implications -- for example, -a relay with a tiny exit policy won't get the Exit flag, and thus won't -ever get picked as an exit relay. Plus, our "enclave exit" model is out -the window unless we figure out a cool trick. - -More generally, we'll probably want to compress the descriptors that we -send back; maybe 8k is a good upper bound? I wonder if we could ask for -several descriptors, and bundle back all of the ones that fit in the 8k? - -We'd also want to put the load balancing weights into the networkstatus, -so clients can choose fast nodes more often without needing to see the -descriptors. This is a good opportunity for the authorities to be able -to put "more accurate" weights in if they learn to detect attacks. It -also means we should consider running automated audits to make sure the -authorities aren't trying to snooker everybody. - -I'm aiming to get Peter Palfrader to tackle this problem in mid 2008, -but I bet he could use some help. - --------------------------------------------------------------------- - -Piece two: inter-relay communication uses UDP - -If relays send packets to/from other relays via UDP, they don't need a -new descriptor for each such link. Thus we'll still need to keep state -for each link, but we won't max out on sockets. - -Clearly a lot more work needs to be done here. Ian Goldberg has a student -who has been working on it, and if all goes well we'll be chipping in -some funding to continue that. Also, Camilo Viecco has been doing his -PhD thesis on it. - --------------------------------------------------------------------- - -Piece three: networkstatus documents get partitioned - -While the authorities should be expected to be able to handle learning -about all the relays, there's no reason the clients or the mirrors need -to. Authorities should put a cap on the number of relays listed in a -single networkstatus, and split them when they get too big. - -We'd need a good way to have each authority come to the same conclusion -about which partition a given relay goes into. - -Directory mirrors would then mirror all the relay descriptors in their -partition. This is compatible with 'piece one' above, since clients in -a given partition will only ask about descriptors in that partition. - -More complex versions of this design would involve overlapping partitions, -but that would seem to start contradicting other parts of this proposal -right quick. - -Nobody is working on this piece yet. It's hard to say when we'll need -it, but it would be nice to have some more thought on it before the week -that we need it. - --------------------------------------------------------------------- - diff --git a/doc/spec/proposals/ideas/xxx-hide-platform.txt b/doc/spec/proposals/ideas/xxx-hide-platform.txt deleted file mode 100644 index 3fed5cfbd..000000000 --- a/doc/spec/proposals/ideas/xxx-hide-platform.txt +++ /dev/null @@ -1,39 +0,0 @@ -Filename: xxx-hide-platform.txt -Title: Hide Tor Platform Information -Version: $Revision$ -Last-Modified: $Date$ -Author: Jacob Appelbaum -Created: 24-July-2008 -Status: Draft - - - Hiding Tor Platform Information - -0.0 Introduction - -The current Tor program publishes its specific Tor version and related OS -platform information. This information could be misused by an attacker. - -0.1 Current Implementation - -Currently, the Tor binary sends data that looks like the following: - - Tor 0.2.0.26-rc (r14597) on Darwin Power Macintosh - Tor 0.1.2.19 on Windows XP Service Pack 3 [workstation] {terminal services, - single user} - -1.0 Suggested changes - -It would be useful to allow a user to configure the disclosure of such -information. Such a change would be an option in the torrc file like so: - - HidePlatform Yes - -1.1 Suggested default behavior in the future - -If a user would like to disclose this information, they could configure their -Tor to do so. - - HidePlatform No - - diff --git a/doc/spec/proposals/ideas/xxx-port-knocking.txt b/doc/spec/proposals/ideas/xxx-port-knocking.txt deleted file mode 100644 index 9fbcdf354..000000000 --- a/doc/spec/proposals/ideas/xxx-port-knocking.txt +++ /dev/null @@ -1,93 +0,0 @@ -Filename: xxx-port-knocking.txt -Title: Port knocking for bridge scanning resistance -Version: $Revision$ -Last-Modified: $Date$ -Author: Jacob Appelbaum -Created: 19-April-2009 -Status: Draft - - Port knocking for bridge scanning resistance - -0.0 Introduction - -This document is a collection of ideas relating to improving scanning -resistance for private bridge relays. This is intented to stop opportunistic -network scanning and subsequent discovery of private bridge relays. - - -0.1 Current Implementation - -Currently private bridges are only hidden by their obscurity. If you know -a bridge ip address, the bridge can be detected trivially and added to a block -list. - -0.2 Configuring an external port knocking program to control the firewall - -It is currently possible for bridge operators to configure a port knocking -daemon that controls access to the incoming OR port. This is currently out of -scope for Tor and Tor configuration. This process requires the firewall to know -the current nodes in the Tor network. - -1.0 Suggested changes - -Private bridge operators should be able to configure a method of hiding their -relay. Only authorized users should be able to communicate with the private -bridge. This should be done with Tor and if possible without the help of the -firewall. It should be possible for a Tor user to enter a secret key into -Tor or optionally Vidalia on a per bridge basis. This secret key should be -used to authenticate the bridge user to the private bridge. - -1.x Issues with low ports and bind() for ORPort - -Tor opens low numbered ports during startup and then drops privileges. It is -no longer possible to rebind to those lower ports after they are closed. - -1.x Issues with OS level packet filtering - -Tor does not know about any OS level packet filtering. Currently there is no -packet filters that understands the Tor network in real time. - -1.x Possible partioning of users by bridge operator - -Depending on implementation, it may be possible for bridge operators to -uniquely identify users. This appears to be a general bridge issue when a -bridge operator uniquely deploys bridges per user. - -2.0 Implementation ideas - -This is a suggested set of methods for port knocking. - -2.x Using SPA port knocking - -Single Packet Authentication port knocking encodes all required data into a -single UDP packet. Improperly formatted packets may be simply discarded. -Properly formatted packets should be processed and appropriate actions taken. - -2.x Using DNS as a transport for SPA - -It should be possible for Tor to bind to port 53 at startup and merely drop all -packets that are not valid. UDP does not require a response and invalid packets -will not trigger a response from Tor. With base32 encoding it should be -possible to encode SPA as valid DNS requests. This should allow use of the -public DNS infrastructure for authorization requests if desired. - -2.x Ghetto firewalling with opportunistic connection closing - -Until a user has authenticated with Tor, Tor only has a UDP listener. This -listener should never send data in response, it should only open an ORPort -when a user has successfully authenticated. After a user has authenticated -with Tor to open an ORPort, only users who have authenticated will be able -to use it. All other users as identified by their ip address will have their -connection closed before any data is sent or received. This should be -accomplished with an access policy. By default, the access policy should block -all access to the ORPort. - -2.x Timing and reset of access policies - -Access to the ORPort is sensitive. The bridge should remove any exceptions -to its access policy regularly when the ORPort is unused. Valid users should -reauthenticate if they do not use the ORPort within a given time frame. - -2.x Additional considerations - -There are many. A format of the packet and the crypto involved is a good start. diff --git a/doc/spec/proposals/ideas/xxx-rate-limit-exits.txt b/doc/spec/proposals/ideas/xxx-rate-limit-exits.txt deleted file mode 100644 index 81fed20af..000000000 --- a/doc/spec/proposals/ideas/xxx-rate-limit-exits.txt +++ /dev/null @@ -1,63 +0,0 @@ - -1. Overview - - We should rate limit the volume of stream creations at exits: - -2.1. Per-circuit limits - - If a given circuit opens more than N streams in X seconds, further - stream requests over the next Y seconds should fail with the reason - 'resourcelimit'. Clients will automatically notice this and switch to - a new circuit. - - The goal is to limit the effects of port scans on a given exit relay, - so the relay's ISP won't get hassled as much. - - First thoughts for parameters would be N=100 streams in X=5 seconds - causes 30 seconds of fails; and N=300 streams in X=30 seconds causes - 30 seconds of fails. - - We could simplify by, instead of having a "for 30 seconds" parameter, - just marking the circuit as forever failing new requests. (We don't want - to just close the circuit because it may still have open streams on it.) - -2.2. Per-destination limits - - If a given circuit opens more than N1 streams in X seconds to a single - IP address, or all the circuits combined open more than N2 streams, - then we should fail further attempts to reach that address for a while. - - The goal is to limit the abuse that Tor exit relays can dish out - to a single target either for socket DoS or for web crawling, in - the hopes of a) not triggering their automated defenses, and b) not - making them upset at Tor. Hopefully these self-imposed bans will be - much shorter-lived than bans or barriers put up by the websites. - -3. Issues - -3.1. Circuit-creation overload - - Making clients move to new circuits more often will cause more circuit - creation requests. - -3.2. How to pick the parameters? - - If we pick the numbers too low, then popular sites are effectively - cut out of Tor. If we pick them too high, we don't do much good. - - Worse, picking them wrong isn't easy to fix, since the deployed Tor - servers will ship with a certain set of numbers. - - We could put numbers (or "general settings") in the networkstatus - consensus, and Tor exits would adapt more dynamically. - - We could also have a local config option about how aggressive this - server should be with its parameters. - -4. Client-side limitations - - Perhaps the clients should have built-in rate limits too, so they avoid - harrassing the servers by default? - - Tricky if we want to get Tor clients in use at large enclaves. - diff --git a/doc/spec/proposals/ideas/xxx-separate-streams-by-port.txt b/doc/spec/proposals/ideas/xxx-separate-streams-by-port.txt deleted file mode 100644 index cebde65a9..000000000 --- a/doc/spec/proposals/ideas/xxx-separate-streams-by-port.txt +++ /dev/null @@ -1,61 +0,0 @@ -Filename: xxx-separate-streams-by-port.txt -Title: Separate streams across circuits by destination port -Version: $Revision$ -Last-Modified: $Date$ -Author: Robert Hogan -Created: 21-Oct-2008 -Status: Draft - -Here's a patch Robert Hogan wrote to use only one destination port per -circuit. It's based on a wishlist item Roger wrote, to never send AIM -usernames over the same circuit that we're hoping to browse anonymously -through. The remaining open question is: how many extra circuits does this -cause an ordinary user to create? My guess is not very many, but I'm wary -of putting this in until we have some better estimate. On the other hand, -not putting it in means that we have a known security flaw. Hm. - -Index: src/or/or.h -=================================================================== ---- src/or/or.h (revision 17143) -+++ src/or/or.h (working copy) -@@ -1874,6 +1874,7 @@ - - uint8_t state; /**< Current status of this circuit. */ - uint8_t purpose; /**< Why are we creating this circuit? */ -+ uint16_t service; /**< Port conn must have to use this circuit. */ - - /** How many relay data cells can we package (read from edge streams) - * on this circuit before we receive a circuit-level sendme cell asking -Index: src/or/circuituse.c -=================================================================== ---- src/or/circuituse.c (revision 17143) -+++ src/or/circuituse.c (working copy) -@@ -62,10 +62,16 @@ - return 0; - } - -- if (purpose == CIRCUIT_PURPOSE_C_GENERAL) -+ if (purpose == CIRCUIT_PURPOSE_C_GENERAL) { - if (circ->timestamp_dirty && - circ->timestamp_dirty+get_options()->MaxCircuitDirtiness <= now) - return 0; -+ /* If the circuit is dirty and used for services on another port, -+ then it is not suitable. */ -+ if (circ->service && conn->socks_request->port && -+ (circ->service != conn->socks_request->port)) -+ return 0; -+ } - - /* decide if this circ is suitable for this conn */ - -@@ -1351,7 +1357,9 @@ - if (connection_ap_handshake_send_resolve(conn) < 0) - return -1; - } -- -+ if (conn->socks_request->port -+ && (TO_CIRCUIT(circ)->purpose == CIRCUIT_PURPOSE_C_GENERAL)) -+ TO_CIRCUIT(circ)->service = conn->socks_request->port; - return 1; - } - diff --git a/doc/spec/proposals/ideas/xxx-what-uses-sha1.txt b/doc/spec/proposals/ideas/xxx-what-uses-sha1.txt deleted file mode 100644 index 9b6e20c58..000000000 --- a/doc/spec/proposals/ideas/xxx-what-uses-sha1.txt +++ /dev/null @@ -1,140 +0,0 @@ -Filename: xxx-what-uses-sha1.txt -Title: Where does Tor use SHA-1 today? -Version: $Revision$ -Last-Modified: $Date$ -Author: Nick Mathewson -Created: 30-Dec-2008 -Status: Meta - - -Introduction: - - Tor uses SHA-1 as a message digest. SHA-1 is showing its age: - theoretical attacks for finding collisions against it get better - every year or two, and it will likely be broken in practice before - too long. - - According to smart crypto people, the SHA-2 functions (SHA-256, etc) - share too much of SHA-1's structure to be very good. Some people - like other hash functions; most of these have not seen enough - analysis to be widely regarded as an extra-good idea. - - By 2012, the NIST SHA-3 competition will be done, and with luck we'll - have something good to switch too. But it's probably a bad idea to - wait until 2012 to figure out _how_ to migrate to a new hash - function, for two reasons: - 1) It's not inconceivable we'll want to migrate in a hurry - some time before then. - 2) It's likely that migrating to a new hash function will - require protocol changes, and it's easiest to make protocol - changes backward compatible if we lay the groundwork in - advance. It would suck to have to break compatibility with - a big hard-to-test "flag day" protocol change. - - This document attempts to list everything Tor uses SHA-1 for today. - This is the first step in getting all the design work done to switch - to something else. - - This document SHOULD NOT be a clearinghouse of what to do about our - use of SHA-1. That's better left for other individual proposals. - - -Why now? - - The recent publication of "MD5 considered harmful today: Creating a - rogue CA certificate" by Alexander Sotirov, Marc Stevens, Jacob - Appelbaum, Arjen Lenstra, David Molnar, Dag Arne Osvik, and Benne de - Weger has reminded me that: - - * You can't rely on theoretical attacks to stay theoretical. - * It's quite unpleasant when theoretical attacks become practical - and public on days you were planning to leave for vacation. - * Broken hash functions (which SHA-1 is not quite yet AFAIU) - should be dropped like hot potatoes. Failure to do so can make - one look silly. - - - -What Tor uses hashes for today: - -1. Infrastructure. - - A. Our X.509 certificates are signed with SHA-1. - B. TLS uses SHA-1 (and MD5) internally to generate keys. - C. Some of the TLS ciphersuites we allow use SHA-1. - D. When we sign our code with GPG, it might be using SHA-1. - E. Our GPG keys might be authenticated with SHA-1. - F. OpenSSL's random number generator uses SHA-1, I believe. - -2. The Tor protocol - - A. Everything we sign, we sign using SHA-1-based OAEP-MGF1. - B. Our CREATE cell format uses SHA-1 for: OAEP padding. - C. Our EXTEND cells use SHA-1 to hash the identity key of the - target server. - D. Our CREATED cells use SHA-1 to hash the derived key data. - E. The data we use in CREATE_FAST cells to generate a key is the - length of a SHA-1. - F. The data we send back in a CREATED/CREATED_FAST cell is the length - of a SHA-1. - G. We use SHA-1 to derive our circuit keys from the negotiated g^xy value. - H. We use SHA-1 to derive the digest field of each RELAY cell, but that's - used more as a checksum than as a strong digest. - -3. Directory services - - A. All signatures are generated on the SHA-1 of their corresponding - documents, using PKCS1 padding. - B. Router descriptors identify their corresponding extra-info documents - by their SHA-1 digest. - C. Fingerprints in router descriptors are taken using SHA-1. - D. Fingerprints in authority certs are taken using SHA-1. - E. Fingerprints in dir-source lines of votes and consensuses are taken - using SHA-1. - F. Networkstatuses refer to routers identity keys and descriptors by their - SHA-1 digests. - G. Directory-signature lines identify which key is doing the signing by - the SHA-1 digests of the authority's signing key and its identity key. - H. The following items are downloaded by the SHA-1 of their contents: - XXXX list them - I. The following items are downloaded by the SHA-1 of an identity key: - XXXX list them too. - -4. The rendezvous protocol - - A. Hidden servers use SHA-1 to establish introduction points on relays, - and relays use SHA-1 to check incoming introduction point - establishment requests. - B. Hidden servers use SHA-1 in multiple places when generating hidden - service descriptors. - C. Hidden servers performing basic-type client authorization for their - services use SHA-1 when encrypting introduction points contained in - hidden service descriptors. - D. Hidden service directories use SHA-1 to check whether a given hidden - service descriptor may be published under a given descriptor - identifier or not. - E. Hidden servers use SHA-1 to derive .onion addresses of their - services. - F. Clients use SHA-1 to generate the current hidden service descriptor - identifiers for a given .onion address. - G. Hidden servers use SHA-1 to remember digests of the first parts of - Diffie-Hellman handshakes contained in introduction requests in order - to detect replays. - H. Hidden servers use SHA-1 during the Diffie-Hellman key exchange with - a connecting client. - -5. The bridge protocol - - XXXX write me - -6. The Tor user interface - - A. We log information about servers based on SHA-1 hashes of their - identity keys. - B. The controller identifies servers based on SHA-1 hashes of their - identity keys. - C. Nearly all of our configuration options that list servers allow SHA-1 - hashes of their identity keys. - E. The deprecated .exit notation uses SHA-1 hashes of identity keys - - diff --git a/doc/spec/proposals/reindex.py b/doc/spec/proposals/reindex.py deleted file mode 100755 index 2b4c02516..000000000 --- a/doc/spec/proposals/reindex.py +++ /dev/null @@ -1,117 +0,0 @@ -#!/usr/bin/python - -import re, os -class Error(Exception): pass - -STATUSES = """DRAFT NEEDS-REVISION NEEDS-RESEARCH OPEN ACCEPTED META FINISHED - CLOSED SUPERSEDED DEAD""".split() -REQUIRED_FIELDS = [ "Filename", "Status", "Title" ] -CONDITIONAL_FIELDS = { "OPEN" : [ "Target" ], - "ACCEPTED" : [ "Target "], - "CLOSED" : [ "Implemented-In" ], - "FINISHED" : [ "Implemented-In" ] } -FNAME_RE = re.compile(r'^(\d\d\d)-.*[^\~]$') -DIR = "." -OUTFILE = "000-index.txt" -TMPFILE = OUTFILE+".tmp" - -def indexed(seq): - n = 0 - for i in seq: - yield n, i - n += 1 - -def readProposal(fn): - fields = { } - f = open(fn, 'r') - lastField = None - try: - for lineno, line in indexed(f): - line = line.rstrip() - if not line: - return fields - if line[0].isspace(): - fields[lastField] += " %s"%(line.strip()) - else: - parts = line.split(":", 1) - if len(parts) != 2: - raise Error("%s:%s: Neither field nor continuation"% - (fn,lineno)) - else: - fields[parts[0]] = parts[1].strip() - lastField = parts[0] - - return fields - finally: - f.close() - -def checkProposal(fn, fields): - status = fields.get("Status") - need_fields = REQUIRED_FIELDS + CONDITIONAL_FIELDS.get(status, []) - for f in need_fields: - if not fields.has_key(f): - raise Error("%s has no %s field"%(fn, f)) - if fn != fields['Filename']: - print `fn`, `fields['Filename']` - raise Error("Mismatched Filename field in %s"%fn) - if fields['Title'][-1] == '.': - fields['Title'] = fields['Title'][:-1] - - status = fields['Status'] = status.upper() - if status not in STATUSES: - raise Error("I've never heard of status %s in %s"%(status,fn)) - if status in [ "SUPERSEDED", "DEAD" ]: - for f in [ 'Implemented-In', 'Target' ]: - if fields.has_key(f): del fields[f] - -def readProposals(): - res = [] - for fn in os.listdir(DIR): - m = FNAME_RE.match(fn) - if not m: continue - if not fn.endswith(".txt"): - raise Error("%s doesn't end with .txt"%fn) - num = m.group(1) - fields = readProposal(fn) - checkProposal(fn, fields) - fields['num'] = num - res.append(fields) - return res - -def writeIndexFile(proposals): - proposals.sort(key=lambda f:f['num']) - seenStatuses = set() - for p in proposals: - seenStatuses.add(p['Status']) - - out = open(TMPFILE, 'w') - inf = open(OUTFILE, 'r') - for line in inf: - out.write(line) - if line.startswith("====="): break - inf.close() - - out.write("Proposals by number:\n\n") - for prop in proposals: - out.write("%(num)s %(Title)s [%(Status)s]\n"%prop) - out.write("\n\nProposals by status:\n\n") - for s in STATUSES: - if s not in seenStatuses: continue - out.write(" %s:\n"%s) - for prop in proposals: - if s == prop['Status']: - out.write(" %(num)s %(Title)s"%prop) - if prop.has_key('Target'): - out.write(" [for %(Target)s]"%prop) - if prop.has_key('Implemented-In'): - out.write(" [in %(Implemented-In)s]"%prop) - out.write("\n") - out.close() - os.rename(TMPFILE, OUTFILE) - -try: - os.unlink(TMPFILE) -except OSError: - pass - -writeIndexFile(readProposals()) diff --git a/doc/spec/rend-spec.txt b/doc/spec/rend-spec.txt deleted file mode 100644 index e3fbe2253..000000000 --- a/doc/spec/rend-spec.txt +++ /dev/null @@ -1,768 +0,0 @@ -$Id$ - - Tor Rendezvous Specification - -0. Overview and preliminaries - - Read - https://www.torproject.org/doc/design-paper/tor-design.html#sec:rendezvous - before you read this specification. It will make more sense. - - Rendezvous points provide location-hidden services (server - anonymity) for the onion routing network. With rendezvous points, - Bob can offer a TCP service (say, a webserver) via the onion - routing network, without revealing the IP of that service. - - Bob does this by anonymously advertising a public key for his - service, along with a list of onion routers to act as "Introduction - Points" for his service. He creates forward circuits to those - introduction points, and tells them about his public key. To - connect to Bob, Alice first builds a circuit to an OR to act as - her "Rendezvous Point." She then connects to one of Bob's chosen - introduction points, optionally provides authentication or - authorization information, and asks it to tell him about her Rendezvous - Point (RP). If Bob chooses to answer, he builds a circuit to her - RP, and tells it to connect him to Alice. The RP joins their - circuits together, and begins relaying cells. Alice's 'BEGIN' - cells are received directly by Bob's OP, which passes data to - and from the local server implementing Bob's service. - - Below we describe a network-level specification of this service, - along with interfaces to make this process transparent to Alice - (so long as she is using an OP). - -0.1. Notation, conventions and prerequisites - - In the specifications below, we use the same notation and terminology - as in "tor-spec.txt". The service specified here also requires the - existence of an onion routing network as specified in that file. - - H(x) is a SHA1 digest of x. - PKSign(SK,x) is a PKCS.1-padded RSA signature of x with SK. - PKEncrypt(SK,x) is a PKCS.1-padded RSA encryption of x with SK. - Public keys are all RSA, and encoded in ASN.1. - All integers are stored in network (big-endian) order. - All symmetric encryption uses AES in counter mode, except where - otherwise noted. - - In all discussions, "Alice" will refer to a user connecting to a - location-hidden service, and "Bob" will refer to a user running a - location-hidden service. - - An OP is (as defined elsewhere) an "Onion Proxy" or Tor client. - - An OR is (as defined elsewhere) an "Onion Router" or Tor server. - - An "Introduction point" is a Tor server chosen to be Bob's medium-term - 'meeting place'. A "Rendezvous point" is a Tor server chosen by Alice to - be a short-term communication relay between her and Bob. All Tor servers - potentially act as introduction and rendezvous points. - -0.2. Protocol outline - - 1. Bob->Bob's OP: "Offer IP:Port as - public-key-name:Port". [configuration] - (We do not specify this step; it is left to the implementor of - Bob's OP.) - - 2. Bob's OP generates keypair and rendezvous service descriptor: - "Meet public-key X at introduction point A, B, or C." (signed) - - 3. Bob's OP->Introduction point via Tor: [introduction setup] - "This pk is me." - - 4. Bob's OP->directory service via Tor: publishes Bob's service - descriptor [advertisement] - - 5. Out of band, Alice receives a [x.y.]z.onion:port address. - She opens a SOCKS connection to her OP, and requests - x.y.z.onion:port. - - 6. Alice's OP retrieves Bob's descriptor via Tor. [descriptor lookup.] - - 7. Alice's OP chooses a rendezvous point, opens a circuit to that - rendezvous point, and establishes a rendezvous circuit. [rendezvous - setup.] - - 8. Alice connects to the Introduction point via Tor, and tells it about - her rendezvous point and optional authentication/authorization - information. (Encrypted to Bob.) [Introduction 1] - - 9. The Introduction point passes this on to Bob's OP via Tor, along the - introduction circuit. [Introduction 2] - - 10. Bob's OP decides whether to connect to Alice, and if so, creates a - circuit to Alice's RP via Tor. Establishes a shared circuit. - [Rendezvous.] - - 11. Alice's OP sends begin cells to Bob's OP. [Connection] - -0.3. Constants and new cell types - - Relay cell types - 32 -- RELAY_ESTABLISH_INTRO - 33 -- RELAY_ESTABLISH_RENDEZVOUS - 34 -- RELAY_INTRODUCE1 - 35 -- RELAY_INTRODUCE2 - 36 -- RELAY_RENDEZVOUS1 - 37 -- RELAY_RENDEZVOUS2 - 38 -- RELAY_INTRO_ESTABLISHED - 39 -- RELAY_RENDEZVOUS_ESTABLISHED - 40 -- RELAY_COMMAND_INTRODUCE_ACK - -0.4. Version overview - - There are several parts in the hidden service protocol that have - changed over time, each of them having its own version number, whereas - other parts remained the same. The following list of potentially - versioned protocol parts should help reduce some confusion: - - - Hidden service descriptor: the binary-based v0 was the default for - a long time, and an ascii-based v2 has been added by proposal - 114. See 1.2. - - - Hidden service descriptor propagation mechanism: currently related to - the hidden service descriptor version -- v0 publishes to the original - hs directory authorities, whereas v2 publishes to a rotating subset - of relays with the "hsdir" flag; see 1.4 and 1.6. - - - Introduction protocol for how to generate an introduction cell: - v0 specified a nickname for the rendezvous point and assumed the - relay would know about it, whereas v2 now specifies IP address, - port, and onion key so the relay doesn't need to already recognize - it. See 1.8. - -1. The Protocol - -1.1. Bob configures his local OP. - - We do not specify a format for the OP configuration file. However, - OPs SHOULD allow Bob to provide more than one advertised service - per OP, and MUST allow Bob to specify one or more virtual ports per - service. Bob provides a mapping from each of these virtual ports - to a local IP:Port pair. - -1.2. Bob's OP generates service descriptors. - - The first time the OP provides an advertised service, it generates - a public/private keypair (stored locally). Periodically, the OP - generates and publishes a descriptor of type "V0". - - The "V0" descriptor contains: - - KL Key length [2 octets] - PK Bob's public key [KL octets] - TS A timestamp [4 octets] - NI Number of introduction points [2 octets] - Ipt A list of NUL-terminated ORs [variable] - SIG Signature of above fields [variable] - - KL is the length of PK, in octets. - TS is the number of seconds elapsed since Jan 1, 1970. - - The members of Ipt may be either (a) nicknames, or (b) identity key - digests, encoded in hex, and prefixed with a '$'. Clients must - accept both forms. Services must only generate the second form. - Once 0.0.9.x is obsoleted, we can drop the first form. - - [It's ok for Bob to advertise 0 introduction points. He might want - to do that if he previously advertised some introduction points, - and now he doesn't have any. -RD] - - Beginning with 0.2.0.10-alpha, Bob's OP encodes "V2" descriptors in - addition to "V0" descriptors. The format of a "V2" descriptor is as - follows: - - "rendezvous-service-descriptor" descriptor-id NL - - [At start, exactly once] - - Indicates the beginning of the descriptor. "descriptor-id" is a - periodically changing identifier of 160 bits formatted as 32 base32 - chars that is calculated by the hidden service and its clients. If - the optional "descriptor-cookie" is used, this "descriptor-id" - cannot be computed by anyone else. (Everyone can verify that this - "descriptor-id" belongs to the rest of the descriptor, even without - knowing the optional "descriptor-cookie", as described below.) The - "descriptor-id" is calculated by performing the following operation: - - descriptor-id = - H(permanent-id | H(time-period | descriptor-cookie | replica)) - - "permanent-id" is the permanent identifier of the hidden service, - consisting of 80 bits. It can be calculated by computing the hash value - of the public hidden service key and truncating after the first 80 bits: - - permanent-id = H(public-key)[:10] - - "H(time-period | descriptor-cookie | replica)" is the (possibly - secret) id part that is - necessary to verify that the hidden service is the true originator - of this descriptor. It can only be created by the hidden service - and its clients, but the "signature" below can only be created by - the service. - - "descriptor-cookie" is an optional secret password of 128 bits that - is shared between the hidden service provider and its clients. - - "replica" denotes the number of the non-consecutive replica. - - (Each descriptor is replicated on a number of _consecutive_ nodes - in the identifier ring by making every storing node responsible - for the identifier intervals starting from its 3rd predecessor's - ID to its own ID. In addition to that, every service publishes - multiple descriptors with different descriptor IDs in order to - distribute them to different places on the ring. Therefore, - "replica" chooses one of the _non-consecutive_ replicas. -KL) - - The "time-period" changes periodically depending on the global time and - as a function of "permanent-id". The current value for "time-period" can - be calculated using the following formula: - - time-period = (current-time + permanent-id-byte * 86400 / 256) - / 86400 - - "current-time" contains the current system time in seconds since - 1970-01-01 00:00, e.g. 1188241957. "permanent-id-byte" is the first - (unsigned) byte of the permanent identifier (which is in network - order), e.g. 143. Adding the product of "permanent-id-byte" and - 86400 (seconds per day), divided by 256, prevents "time-period" from - changing for all descriptors at the same time of the day. The result - of the overall operation is a (network-ordered) 32-bit integer, e.g. - 13753 or 0x000035B9 with the example values given above. - - "version" version-number NL - - [Exactly once] - - The version number of this descriptor's format. In this case: 2. - - "permanent-key" NL a public key in PEM format - - [Exactly once] - - The public key of the hidden service which is required to verify the - "descriptor-id" and the "signature". - - "secret-id-part" secret-id-part NL - - [Exactly once] - - The result of the following operation as explained above, formatted as - 32 base32 chars. Using this secret id part, everyone can verify that - the signed descriptor belongs to "descriptor-id". - - secret-id-part = H(time-period | descriptor-cookie | replica) - - "publication-time" YYYY-MM-DD HH:MM:SS NL - - [Exactly once] - - A timestamp when this descriptor has been created. - - "protocol-versions" version-string NL - - [Exactly once] - - A comma-separated list of recognized and permitted version numbers - for use in INTRODUCE cells; these versions are described in section - 1.8 below. - - "introduction-points" NL encrypted-string - - [At most once] - - A list of introduction points. If the optional "descriptor-cookie" is - used, this list is encrypted with AES in CTR mode with a random - initialization vector of 128 bits that is written to - the beginning of the encrypted string, and the "descriptor-cookie" as - secret key of 128 bits length. - - The string containing the introduction point data (either encrypted - or not) is encoded in base64, and surrounded with - "-----BEGIN MESSAGE-----" and "-----END MESSAGE-----". - - The unencrypted string may begin with: - - ["service-authentication" auth-type NL auth-data ... reserved] - - [At start, any number] - - The service-specific authentication data can be used to perform - client authentication. This data is independent of the selected - introduction point as opposed to "intro-authentication" below. - - Subsequently, an arbitrary number of introduction point entries may - follow, each containing the following data: - - "introduction-point" identifier NL - - [At start, exactly once] - - The identifier of this introduction point: the base-32 encoded - hash of this introduction point's identity key. - - "ip-address" ip-address NL - - [Exactly once] - - The IP address of this introduction point. - - "onion-port" port NL - - [Exactly once] - - The TCP port on which the introduction point is listening for - incoming onion requests. - - "onion-key" NL a public key in PEM format - - [Exactly once] - - The public key that can be used to encrypt messages to this - introduction point. - - "service-key" NL a public key in PEM format - - [Exactly once] - - The public key that can be used to encrypt messages to the hidden - service. - - ["intro-authentication" auth-type NL auth-data ... reserved] - - [Any number] - - The introduction-point-specific authentication data can be used - to perform client authentication. This data depends on the - selected introduction point as opposed to "service-authentication" - above. - - (This ends the fields in the encrypted portion of the descriptor.) - - "signature" NL signature-string - - [At end, exactly once] - - A signature of all fields above with the private key of the hidden - service. - -1.2.1. Other descriptor formats we don't use. - - The V1 descriptor format was understood and accepted from - 0.1.1.5-alpha-cvs to 0.2.0.6-alpha-dev, but no Tors generated it and - it was removed: - - V Format byte: set to 255 [1 octet] - V Version byte: set to 1 [1 octet] - KL Key length [2 octets] - PK Bob's public key [KL octets] - TS A timestamp [4 octets] - PROTO Protocol versions: bitmask [2 octets] - NI Number of introduction points [2 octets] - For each introduction point: (as in INTRODUCE2 cells) - IP Introduction point's address [4 octets] - PORT Introduction point's OR port [2 octets] - ID Introduction point identity ID [20 octets] - KLEN Length of onion key [2 octets] - KEY Introduction point onion key [KLEN octets] - SIG Signature of above fields [variable] - - A hypothetical "V1" descriptor, that has never been used but might - be useful for historical reasons, contains: - - V Format byte: set to 255 [1 octet] - V Version byte: set to 1 [1 octet] - KL Key length [2 octets] - PK Bob's public key [KL octets] - TS A timestamp [4 octets] - PROTO Rendezvous protocol versions: bitmask [2 octets] - NA Number of auth mechanisms accepted [1 octet] - For each auth mechanism: - AUTHT The auth type that is supported [2 octets] - AUTHL Length of auth data [1 octet] - AUTHD Auth data [variable] - NI Number of introduction points [2 octets] - For each introduction point: (as in INTRODUCE2 cells) - ATYPE An address type (typically 4) [1 octet] - ADDR Introduction point's IP address [4 or 16 octets] - PORT Introduction point's OR port [2 octets] - AUTHT The auth type that is supported [2 octets] - AUTHL Length of auth data [1 octet] - AUTHD Auth data [variable] - ID Introduction point identity ID [20 octets] - KLEN Length of onion key [2 octets] - KEY Introduction point onion key [KLEN octets] - SIG Signature of above fields [variable] - - AUTHT specifies which authentication/authorization mechanism is - required by the hidden service or the introduction point. AUTHD - is arbitrary data that can be associated with an auth approach. - Currently only AUTHT of [00 00] is supported, with an AUTHL of 0. - See section 2 of this document for details on auth mechanisms. - -1.3. Bob's OP establishes his introduction points. - - The OP establishes a new introduction circuit to each introduction - point. These circuits MUST NOT be used for anything but hidden service - introduction. To establish the introduction, Bob sends a - RELAY_ESTABLISH_INTRO cell, containing: - - KL Key length [2 octets] - PK Bob's public key [KL octets] - HS Hash of session info [20 octets] - SIG Signature of above information [variable] - - [XXX011, need to add auth information here. -RD] - - To prevent replay attacks, the HS field contains a SHA-1 hash based on the - shared secret KH between Bob's OP and the introduction point, as - follows: - HS = H(KH | "INTRODUCE") - That is: - HS = H(KH | [49 4E 54 52 4F 44 55 43 45]) - (KH, as specified in tor-spec.txt, is H(g^xy | [00]) .) - - Upon receiving such a cell, the OR first checks that the signature is - correct with the included public key. If so, it checks whether HS is - correct given the shared state between Bob's OP and the OR. If either - check fails, the OP discards the cell; otherwise, it associates the - circuit with Bob's public key, and dissociates any other circuits - currently associated with PK. On success, the OR sends Bob a - RELAY_INTRO_ESTABLISHED cell with an empty payload. - - If a hidden service is configured to publish only v2 hidden service - descriptors, Bob's OP does not include its own public key in the - RELAY_ESTABLISH_INTRO cell, but the public key of a freshly generated - key pair. The OP also includes these fresh public keys in the v2 hidden - service descriptor together with the other introduction point - information. The reason is that the introduction point does not need to - and therefore should not know for which hidden service it works, so as - to prevent it from tracking the hidden service's activity. If the hidden - service is configured to publish both, v0 and v2 descriptors, two - separate sets of introduction points are established. - -1.4. Bob's OP advertises his service descriptor(s). - - Bob's OP opens a stream to each directory server's directory port via Tor. - (He may re-use old circuits for this.) Over this stream, Bob's OP makes - an HTTP 'POST' request, to a URL "/tor/rendezvous/publish" relative to the - directory server's root, containing as its body Bob's service descriptor. - - Bob should upload a service descriptor for each version format that - is supported in the current Tor network. - - Upon receiving a descriptor, the directory server checks the signature, - and discards the descriptor if the signature does not match the enclosed - public key. Next, the directory server checks the timestamp. If the - timestamp is more than 24 hours in the past or more than 1 hour in the - future, or the directory server already has a newer descriptor with the - same public key, the server discards the descriptor. Otherwise, the - server discards any older descriptors with the same public key and - version format, and associates the new descriptor with the public key. - The directory server remembers this descriptor for at least 24 hours - after its timestamp. At least every 18 hours, Bob's OP uploads a - fresh descriptor. - - If Bob's OP is configured to publish v2 descriptors instead of or in - addition to v0 descriptors, it does so to a changing subset of all v2 - hidden service directories instead of the authoritative directory - servers. Therefore, Bob's OP opens a stream via Tor to each - responsible hidden service directory. (He may re-use old circuits - for this.) Over this stream, Bob's OP makes an HTTP 'POST' request to a - URL "/tor/rendezvous2/publish" relative to the hidden service - directory's root, containing as its body Bob's service descriptor. - - At any time, there are 6 hidden service directories responsible for - keeping replicas of a descriptor; they consist of 2 sets of 3 hidden - service directories with consecutive onion IDs. Bob's OP learns about - the complete list of hidden service directories by filtering the - consensus status document received from the directory authorities. A - hidden service directory is deemed responsible for all descriptor IDs in - the interval from its direct predecessor, exclusive, to its own ID, - inclusive; it further holds replicas for its 2 predecessors. A - participant only trusts its own routing list and never learns about - routing information from other parties. - - Bob's OP publishes a new v2 descriptor once an hour or whenever its - content changes. V2 descriptors can be found by clients within a given - time period of 24 hours, after which they change their ID as described - under 1.2. If a published descriptor would be valid for less than 60 - minutes (= 2 x 30 minutes to allow the server to be 30 minutes behind - and the client 30 minutes ahead), Bob's OP publishes the descriptor - under the ID of both, the current and the next publication period. - -1.5. Alice receives a x.y.z.onion address. - - When Alice receives a pointer to a location-hidden service, it is as a - hostname of the form "z.onion" or "y.z.onion" or "x.y.z.onion", where - z is a base-32 encoding of a 10-octet hash of Bob's service's public - key, computed as follows: - - 1. Let H = H(PK). - 2. Let H' = the first 80 bits of H, considering each octet from - most significant bit to least significant bit. - 2. Generate a 16-character encoding of H', using base32 as defined - in RFC 3548. - - (We only use 80 bits instead of the 160 bits from SHA1 because we - don't need to worry about arbitrary collisions, and because it will - make handling the url's more convenient.) - - The string "x", if present, is the base-32 encoding of the - authentication/authorization required by the introduction point. - The string "y", if present, is the base-32 encoding of the - authentication/authorization required by the hidden service. - Omitting a string is taken to mean auth type [00 00]. - See section 2 of this document for details on auth mechanisms. - - [Yes, numbers are allowed at the beginning. See RFC 1123. -NM] - -1.6. Alice's OP retrieves a service descriptor. - - Alice opens a stream to a directory server via Tor, and makes an HTTP GET - request for the document '/tor/rendezvous/', where '' is replaced - with the encoding of Bob's public key as described above. (She may re-use - old circuits for this.) The directory replies with a 404 HTTP response if - it does not recognize , and otherwise returns Bob's most recently - uploaded service descriptor. - - If Alice's OP receives a 404 response, it tries the other directory - servers, and only fails the lookup if none recognize the public key hash. - - Upon receiving a service descriptor, Alice verifies with the same process - as the directory server uses, described above in section 1.4. - - The directory server gives a 400 response if it cannot understand Alice's - request. - - Alice should cache the descriptor locally, but should not use - descriptors that are more than 24 hours older than their timestamp. - [Caching may make her partitionable, but she fetched it anonymously, - and we can't very well *not* cache it. -RD] - - Alice's OP fetches v2 descriptors in parallel to v0 descriptors. Similarly - to the description in section 1.4, the OP fetches a v2 descriptor from a - randomly chosen hidden service directory out of the changing subset of - 6 nodes. If the request is unsuccessful, Alice retries the other - remaining responsible hidden service directories in a random order. - Alice relies on Bob to care about a potential clock skew between the two - by possibly storing two sets of descriptors (see end of section 1.4). - - Alice's OP opens a stream via Tor to the chosen v2 hidden service - directory. (She may re-use old circuits for this.) Over this stream, - Alice's OP makes an HTTP 'GET' request for the document - "/tor/rendezvous2/", where z is replaced with the encoding of the - descriptor ID. The directory replies with a 404 HTTP response if it does - not recognize , and otherwise returns Bob's most recently uploaded - service descriptor. - -1.7. Alice's OP establishes a rendezvous point. - - When Alice requests a connection to a given location-hidden service, - and Alice's OP does not have an established circuit to that service, - the OP builds a rendezvous circuit. It does this by establishing - a circuit to a randomly chosen OR, and sending a - RELAY_ESTABLISH_RENDEZVOUS cell to that OR. The body of that cell - contains: - - RC Rendezvous cookie [20 octets] - - [XXX011 this looks like an auth mechanism. should we generalize here? -RD] - - The rendezvous cookie is an arbitrary 20-byte value, chosen randomly by - Alice's OP. - - Upon receiving a RELAY_ESTABLISH_RENDEZVOUS cell, the OR associates the - RC with the circuit that sent it. It replies to Alice with an empty - RELAY_RENDEZVOUS_ESTABLISHED cell to indicate success. - - Alice's OP MUST NOT use the circuit which sent the cell for any purpose - other than rendezvous with the given location-hidden service. - -1.8. Introduction: from Alice's OP to Introduction Point - - Alice builds a separate circuit to one of Bob's chosen introduction - points, and sends it a RELAY_INTRODUCE1 cell containing: - - Cleartext - PK_ID Identifier for Bob's PK [20 octets] - Encrypted to Bob's PK: (in the v0 intro protocol) - RP Rendezvous point's nickname [20 octets] - RC Rendezvous cookie [20 octets] - g^x Diffie-Hellman data, part 1 [128 octets] - OR (in the v1 intro protocol) - VER Version byte: set to 1. [1 octet] - RP Rendezvous point nick or ID [42 octets] - RC Rendezvous cookie [20 octets] - g^x Diffie-Hellman data, part 1 [128 octets] - OR (in the v2 intro protocol) - VER Version byte: set to 2. [1 octet] - IP Rendezvous point's address [4 octets] - PORT Rendezvous point's OR port [2 octets] - ID Rendezvous point identity ID [20 octets] - KLEN Length of onion key [2 octets] - KEY Rendezvous point onion key [KLEN octets] - RC Rendezvous cookie [20 octets] - g^x Diffie-Hellman data, part 1 [128 octets] - - PK_ID is the hash of Bob's public key. RP is NUL-padded and - terminated. In version 0, it must contain a nickname. In version 1, - it must contain EITHER a nickname or an identity key digest that is - encoded in hex and prefixed with a '$'. - - The hybrid encryption to Bob's PK works just like the hybrid - encryption in CREATE cells (see tor-spec). Thus the payload of the - version 0 RELAY_INTRODUCE1 cell on the wire will contain - 20+42+16+20+20+128=246 bytes, and the version 1 and version 2 - introduction formats have other sizes. - - Through Tor 0.2.0.6-alpha, clients only generated the v0 introduction - format, whereas hidden services have understood and accepted v0, - v1, and v2 since 0.1.1.x. As of Tor 0.2.0.7-alpha and 0.1.2.18, - clients switched to using the v2 intro format. - - If Alice has downloaded a v2 descriptor, she uses the contained public - key ("service-key") instead of Bob's public key to create the - RELAY_INTRODUCE1 cell as described above. - -1.8.1. Other introduction formats we don't use. - - We briefly speculated about using the following format for the - "encrypted to Bob's PK" part of the introduction, but no Tors have - ever generated these. - - VER Version byte: set to 3. [1 octet] - ATYPE An address type (typically 4) [1 octet] - ADDR Rendezvous point's IP address [4 or 16 octets] - PORT Rendezvous point's OR port [2 octets] - AUTHT The auth type that is supported [2 octets] - AUTHL Length of auth data [1 octet] - AUTHD Auth data [variable] - ID Rendezvous point identity ID [20 octets] - KLEN Length of onion key [2 octets] - KEY Rendezvous point onion key [KLEN octets] - RC Rendezvous cookie [20 octets] - g^x Diffie-Hellman data, part 1 [128 octets] - -1.9. Introduction: From the Introduction Point to Bob's OP - - If the Introduction Point recognizes PK_ID as a public key which has - established a circuit for introductions as in 1.3 above, it sends the body - of the cell in a new RELAY_INTRODUCE2 cell down the corresponding circuit. - (If the PK_ID is unrecognized, the RELAY_INTRODUCE1 cell is discarded.) - - After sending the RELAY_INTRODUCE2 cell, the OR replies to Alice with an - empty RELAY_COMMAND_INTRODUCE_ACK cell. If no RELAY_INTRODUCE2 cell can - be sent, the OR replies to Alice with a non-empty cell to indicate an - error. (The semantics of the cell body may be determined later; the - current implementation sends a single '1' byte on failure.) - - When Bob's OP receives the RELAY_INTRODUCE2 cell, it decrypts it with - the private key for the corresponding hidden service, and extracts the - rendezvous point's nickname, the rendezvous cookie, and the value of g^x - chosen by Alice. - -1.10. Rendezvous - - Bob's OP builds a new Tor circuit ending at Alice's chosen rendezvous - point, and sends a RELAY_RENDEZVOUS1 cell along this circuit, containing: - RC Rendezvous cookie [20 octets] - g^y Diffie-Hellman [128 octets] - KH Handshake digest [20 octets] - - (Bob's OP MUST NOT use this circuit for any other purpose.) - - If the RP recognizes RC, it relays the rest of the cell down the - corresponding circuit in a RELAY_RENDEZVOUS2 cell, containing: - - g^y Diffie-Hellman [128 octets] - KH Handshake digest [20 octets] - - (If the RP does not recognize the RC, it discards the cell and - tears down the circuit.) - - When Alice's OP receives a RELAY_RENDEZVOUS2 cell on a circuit which - has sent a RELAY_ESTABLISH_RENDEZVOUS cell but which has not yet received - a reply, it uses g^y and H(g^xy) to complete the handshake as in the Tor - circuit extend process: they establish a 60-octet string as - K = SHA1(g^xy | [00]) | SHA1(g^xy | [01]) | SHA1(g^xy | [02]) - and generate - KH = K[0..15] - Kf = K[16..31] - Kb = K[32..47] - - Subsequently, the rendezvous point passes relay cells, unchanged, from - each of the two circuits to the other. When Alice's OP sends - RELAY cells along the circuit, it first encrypts them with the - Kf, then with all of the keys for the ORs in Alice's side of the circuit; - and when Alice's OP receives RELAY cells from the circuit, it decrypts - them with the keys for the ORs in Alice's side of the circuit, then - decrypts them with Kb. Bob's OP does the same, with Kf and Kb - interchanged. - -1.11. Creating streams - - To open TCP connections to Bob's location-hidden service, Alice's OP sends - a RELAY_BEGIN cell along the established circuit, using the special - address "", and a chosen port. Bob's OP chooses a destination IP and - port, based on the configuration of the service connected to the circuit, - and opens a TCP stream. From then on, Bob's OP treats the stream as an - ordinary exit connection. - [ Except he doesn't include addr in the connected cell or the end - cell. -RD] - - Alice MAY send multiple RELAY_BEGIN cells along the circuit, to open - multiple streams to Bob. Alice SHOULD NOT send RELAY_BEGIN cells for any - other address along her circuit to Bob; if she does, Bob MUST reject them. - -2. Authentication and authorization. - -Foo. - -3. Hidden service directory operation - - This section has been introduced with the v2 hidden service descriptor - format. It describes all operations of the v2 hidden service descriptor - fetching and propagation mechanism that are required for the protocol - described in section 1 to succeed with v2 hidden service descriptors. - -3.1. Configuring as hidden service directory - - Every onion router that has its directory port open can decide whether it - wants to store and serve hidden service descriptors. An onion router which - is configured as such includes the "hidden-service-dir" flag in its router - descriptors that it sends to directory authorities. - - The directory authorities include a new flag "HSDir" for routers that - decided to provide storage for hidden service descriptors and that - have been running for at least 24 hours. - -3.2. Accepting publish requests - - Hidden service directory nodes accept publish requests for v2 hidden service - descriptors and store them to their local memory. (It is not necessary to - make descriptors persistent, because after restarting, the onion router - would not be accepted as a storing node anyway, because it has not been - running for at least 24 hours.) All requests and replies are formatted as - HTTP messages. Requests are initiated via BEGIN_DIR cells directed to - the router's directory port, and formatted as HTTP POST requests to the URL - "/tor/rendezvous2/publish" relative to the hidden service directory's root, - containing as its body a v2 service descriptor. - - A hidden service directory node parses every received descriptor and only - stores it when it thinks that it is responsible for storing that descriptor - based on its own routing table. See section 1.4 for more information on how - to determine responsibility for a certain descriptor ID. - -3.3. Processing fetch requests - - Hidden service directory nodes process fetch requests for hidden service - descriptors by looking them up in their local memory. (They do not need to - determine if they are responsible for the passed ID, because it does no harm - if they deliver a descriptor for which they are not (any more) responsible.) - All requests and replies are formatted as HTTP messages. Requests are - initiated via BEGIN_DIR cells directed to the router's directory port, - and formatted as HTTP GET requests for the document "/tor/rendezvous2/", - where z is replaced with the encoding of the descriptor ID. - diff --git a/doc/spec/socks-extensions.txt b/doc/spec/socks-extensions.txt deleted file mode 100644 index 8d58987f3..000000000 --- a/doc/spec/socks-extensions.txt +++ /dev/null @@ -1,79 +0,0 @@ -$Id$ -Tor's extensions to the SOCKS protocol - -1. Overview - - The SOCKS protocol provides a generic interface for TCP proxies. Client - software connects to a SOCKS server via TCP, and requests a TCP connection - to another address and port. The SOCKS server establishes the connection, - and reports success or failure to the client. After the connection has - been established, the client application uses the TCP stream as usual. - - Tor supports SOCKS4 as defined in [1], SOCKS4A as defined in [2], and - SOCKS5 as defined in [3]. - - The stickiest issue for Tor in supporting clients, in practice, is forcing - DNS lookups to occur at the OR side: if clients do their own DNS lookup, - the DNS server can learn which addresses the client wants to reach. - SOCKS4 supports addressing by IPv4 address; SOCKS4A is a kludge on top of - SOCKS4 to allow addressing by hostname; SOCKS5 supports IPv4, IPv6, and - hostnames. - -1.1. Extent of support - - Tor supports the SOCKS4, SOCKS4A, and SOCKS5 standards, except as follows: - - BOTH: - - The BIND command is not supported. - - SOCKS4,4A: - - SOCKS4 usernames are ignored. - - SOCKS5: - - The (SOCKS5) "UDP ASSOCIATE" command is not supported. - - IPv6 is not supported in CONNECT commands. - - Only the "NO AUTHENTICATION" (SOCKS5) authentication method [00] is - supported. - -2. Name lookup - - As an extension to SOCKS4A and SOCKS5, Tor implements a new command value, - "RESOLVE" [F0]. When Tor receives a "RESOLVE" SOCKS command, it initiates - a remote lookup of the hostname provided as the target address in the SOCKS - request. The reply is either an error (if the address couldn't be - resolved) or a success response. In the case of success, the address is - stored in the portion of the SOCKS response reserved for remote IP address. - - (We support RESOLVE in SOCKS4 too, even though it is unnecessary.) - - For SOCKS5 only, we support reverse resolution with a new command value, - "RESOLVE_PTR" [F1]. In response to a "RESOLVE_PTR" SOCKS5 command with - an IPv4 address as its target, Tor attempts to find the canonical - hostname for that IPv4 record, and returns it in the "server bound - address" portion of the reply. - (This command was not supported before Tor 0.1.2.2-alpha.) - -3. Other command extensions. - - Tor 0.1.2.4-alpha added a new command value: "CONNECT_DIR" [F2]. - In this case, Tor will open an encrypted direct TCP connection to the - directory port of the Tor server specified by address:port (the port - specified should be the ORPort of the server). It uses a one-hop tunnel - and a "BEGIN_DIR" relay cell to accomplish this secure connection. - - The F2 command value was removed in Tor 0.2.0.10-alpha in favor of a - new use_begindir flag in edge_connection_t. - -4. HTTP-resistance - - Tor checks the first byte of each SOCKS request to see whether it looks - more like an HTTP request (that is, it starts with a "G", "H", or "P"). If - so, Tor returns a small webpage, telling the user that his/her browser is - misconfigured. This is helpful for the many users who mistakenly try to - use Tor as an HTTP proxy instead of a SOCKS proxy. - -References: - [1] http://archive.socks.permeo.com/protocol/socks4.protocol - [2] http://archive.socks.permeo.com/protocol/socks4a.protocol - [3] SOCKS5: RFC1928 - diff --git a/doc/spec/tor-spec.txt b/doc/spec/tor-spec.txt deleted file mode 100644 index a321aa869..000000000 --- a/doc/spec/tor-spec.txt +++ /dev/null @@ -1,993 +0,0 @@ -$Id$ - - Tor Protocol Specification - - Roger Dingledine - Nick Mathewson - -Note: This document aims to specify Tor as implemented in 0.2.1.x. Future -versions of Tor may implement improved protocols, and compatibility is not -guaranteed. Compatibility notes are given for versions 0.1.1.15-rc and -later; earlier versions are not compatible with the Tor network as of this -writing. - -This specification is not a design document; most design criteria -are not examined. For more information on why Tor acts as it does, -see tor-design.pdf. - -0. Preliminaries - -0.1. Notation and encoding - - PK -- a public key. - SK -- a private key. - K -- a key for a symmetric cypher. - - a|b -- concatenation of 'a' and 'b'. - - [A0 B1 C2] -- a three-byte sequence, containing the bytes with - hexadecimal values A0, B1, and C2, in that order. - - All numeric values are encoded in network (big-endian) order. - - H(m) -- a cryptographic hash of m. - -0.2. Security parameters - - Tor uses a stream cipher, a public-key cipher, the Diffie-Hellman - protocol, and a hash function. - - KEY_LEN -- the length of the stream cipher's key, in bytes. - - PK_ENC_LEN -- the length of a public-key encrypted message, in bytes. - PK_PAD_LEN -- the number of bytes added in padding for public-key - encryption, in bytes. (The largest number of bytes that can be encrypted - in a single public-key operation is therefore PK_ENC_LEN-PK_PAD_LEN.) - - DH_LEN -- the number of bytes used to represent a member of the - Diffie-Hellman group. - DH_SEC_LEN -- the number of bytes used in a Diffie-Hellman private key (x). - - HASH_LEN -- the length of the hash function's output, in bytes. - - PAYLOAD_LEN -- The longest allowable cell payload, in bytes. (509) - - CELL_LEN -- The length of a Tor cell, in bytes. - -0.3. Ciphers - - For a stream cipher, we use 128-bit AES in counter mode, with an IV of all - 0 bytes. - - For a public-key cipher, we use RSA with 1024-bit keys and a fixed - exponent of 65537. We use OAEP-MGF1 padding, with SHA-1 as its digest - function. We leave the optional "Label" parameter unset. (For OAEP - padding, see ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-1/pkcs-1v2-1.pdf) - - For Diffie-Hellman, we use a generator (g) of 2. For the modulus (p), we - use the 1024-bit safe prime from rfc2409 section 6.2 whose hex - representation is: - - "FFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD129024E08" - "8A67CC74020BBEA63B139B22514A08798E3404DDEF9519B3CD3A431B" - "302B0A6DF25F14374FE1356D6D51C245E485B576625E7EC6F44C42E9" - "A637ED6B0BFF5CB6F406B7EDEE386BFB5A899FA5AE9F24117C4B1FE6" - "49286651ECE65381FFFFFFFFFFFFFFFF" - - As an optimization, implementations SHOULD choose DH private keys (x) of - 320 bits. Implementations that do this MUST never use any DH key more - than once. - [May other implementations reuse their DH keys?? -RD] - [Probably not. Conceivably, you could get away with changing DH keys once - per second, but there are too many oddball attacks for me to be - comfortable that this is safe. -NM] - - For a hash function, we use SHA-1. - - KEY_LEN=16. - DH_LEN=128; DH_SEC_LEN=40. - PK_ENC_LEN=128; PK_PAD_LEN=42. - HASH_LEN=20. - - When we refer to "the hash of a public key", we mean the SHA-1 hash of the - DER encoding of an ASN.1 RSA public key (as specified in PKCS.1). - - All "random" values should be generated with a cryptographically strong - random number generator, unless otherwise noted. - - The "hybrid encryption" of a byte sequence M with a public key PK is - computed as follows: - 1. If M is less than PK_ENC_LEN-PK_PAD_LEN, pad and encrypt M with PK. - 2. Otherwise, generate a KEY_LEN byte random key K. - Let M1 = the first PK_ENC_LEN-PK_PAD_LEN-KEY_LEN bytes of M, - and let M2 = the rest of M. - Pad and encrypt K|M1 with PK. Encrypt M2 with our stream cipher, - using the key K. Concatenate these encrypted values. - [XXX Note that this "hybrid encryption" approach does not prevent - an attacker from adding or removing bytes to the end of M. It also - allows attackers to modify the bytes not covered by the OAEP -- - see Goldberg's PET2006 paper for details. We will add a MAC to this - scheme one day. -RD] - -0.4. Other parameter values - - CELL_LEN=512 - -1. System overview - - Tor is a distributed overlay network designed to anonymize - low-latency TCP-based applications such as web browsing, secure shell, - and instant messaging. Clients choose a path through the network and - build a ``circuit'', in which each node (or ``onion router'' or ``OR'') - in the path knows its predecessor and successor, but no other nodes in - the circuit. Traffic flowing down the circuit is sent in fixed-size - ``cells'', which are unwrapped by a symmetric key at each node (like - the layers of an onion) and relayed downstream. - -1.1. Keys and names - - Every Tor server has multiple public/private keypairs: - - - A long-term signing-only "Identity key" used to sign documents and - certificates, and used to establish server identity. - - A medium-term "Onion key" used to decrypt onion skins when accepting - circuit extend attempts. (See 5.1.) Old keys MUST be accepted for at - least one week after they are no longer advertised. Because of this, - servers MUST retain old keys for a while after they're rotated. - - A short-term "Connection key" used to negotiate TLS connections. - Tor implementations MAY rotate this key as often as they like, and - SHOULD rotate this key at least once a day. - - Tor servers are also identified by "nicknames"; these are specified in - dir-spec.txt. - -2. Connections - - Connections between two Tor servers, or between a client and a server, - use TLS/SSLv3 for link authentication and encryption. All - implementations MUST support the SSLv3 ciphersuite - "SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA", and SHOULD support the TLS - ciphersuite "TLS_DHE_RSA_WITH_AES_128_CBC_SHA" if it is available. - - There are three acceptable ways to perform a TLS handshake when - connecting to a Tor server: "certificates up-front", "renegotiation", and - "backwards-compatible renegotiation". ("Backwards-compatible - renegotiation" is, as the name implies, compatible with both other - handshake types.) - - Before Tor 0.2.0.21, only "certificates up-front" was supported. In Tor - 0.2.0.21 or later, "backwards-compatible renegotiation" is used. - - In "certificates up-front", the connection initiator always sends a - two-certificate chain, consisting of an X.509 certificate using a - short-term connection public key and a second, self- signed X.509 - certificate containing its identity key. The other party sends a similar - certificate chain. The initiator's ClientHello MUST NOT include any - ciphersuites other than: - TLS_DHE_RSA_WITH_AES_256_CBC_SHA - TLS_DHE_RSA_WITH_AES_128_CBC_SHA - SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA - SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA - - In "renegotiation", the connection initiator sends no certificates, and - the responder sends a single connection certificate. Once the TLS - handshake is complete, the initiator renegotiates the handshake, with each - parties sending a two-certificate chain as in "certificates up-front". - The initiator's ClientHello MUST include at least once ciphersuite not in - the list above. The responder SHOULD NOT select any ciphersuite besides - those in the list above. - [The above "should not" is because some of the ciphers that - clients list may be fake.] - - In "backwards-compatible renegotiation", the connection initiator's - ClientHello MUST include at least one ciphersuite other than those listed - above. The connection responder examines the initiator's ciphersuite list - to see whether it includes any ciphers other than those included in the - list above. If extra ciphers are included, the responder proceeds as in - "renegotiation": it sends a single certificate and does not request - client certificates. Otherwise (in the case that no extra ciphersuites - are included in the ClientHello) the responder proceeds as in - "certificates up-front": it requests client certificates, and sends a - two-certificate chain. In either case, once the responder has sent its - certificate or certificates, the initiator counts them. If two - certificates have been sent, it proceeds as in "certificates up-front"; - otherwise, it proceeds as in "renegotiation". - - All new implementations of the Tor server protocol MUST support - "backwards-compatible renegotiation"; clients SHOULD do this too. If - this is not possible, new client implementations MUST support both - "renegotiation" and "certificates up-front" and use the router's - published link protocols list (see dir-spec.txt on the "protocols" entry) - to decide which to use. - - In all of the above handshake variants, certificates sent in the clear - SHOULD NOT include any strings to identify the host as a Tor server. In - the "renegotation" and "backwards-compatible renegotiation", the - initiator SHOULD chose a list of ciphersuites and TLS extensions chosen - to mimic one used by a popular web browser. - - Responders MUST NOT select any TLS ciphersuite that lacks ephemeral keys, - or whose symmetric keys are less then KEY_LEN bits, or whose digests are - less than HASH_LEN bits. Responders SHOULD NOT select any SSLv3 - ciphersuite other than those listed above. - - Even though the connection protocol is identical, we will think of the - initiator as either an onion router (OR) if it is willing to relay - traffic for other Tor users, or an onion proxy (OP) if it only handles - local requests. Onion proxies SHOULD NOT provide long-term-trackable - identifiers in their handshakes. - - In all handshake variants, once all certificates are exchanged, all - parties receiving certificates must confirm that the identity key is as - expected. (When initiating a connection, the expected identity key is - the one given in the directory; when creating a connection because of an - EXTEND cell, the expected identity key is the one given in the cell.) If - the key is not as expected, the party must close the connection. - - When connecting to an OR, all parties SHOULD reject the connection if that - OR has a malformed or missing certificate. When accepting an incoming - connection, an OR SHOULD NOT reject incoming connections from parties with - malformed or missing certificates. (However, an OR should not believe - that an incoming connection is from another OR unless the certificates - are present and well-formed.) - - [Before version 0.1.2.8-rc, ORs rejected incoming connections from ORs and - OPs alike if their certificates were missing or malformed.] - - Once a TLS connection is established, the two sides send cells - (specified below) to one another. Cells are sent serially. All - cells are CELL_LEN bytes long. Cells may be sent embedded in TLS - records of any size or divided across TLS records, but the framing - of TLS records MUST NOT leak information about the type or contents - of the cells. - - TLS connections are not permanent. Either side MAY close a connection - if there are no circuits running over it and an amount of time - (KeepalivePeriod, defaults to 5 minutes) has passed since the last time - any traffic was transmitted over the TLS connection. Clients SHOULD - also hold a TLS connection with no circuits open, if it is likely that a - circuit will be built soon using that connection. - - (As an exception, directory servers may try to stay connected to all of - the ORs -- though this will be phased out for the Tor 0.1.2.x release.) - - To avoid being trivially distinguished from servers, client-only Tor - instances are encouraged but not required to use a two-certificate chain - as well. Clients SHOULD NOT keep using the same certificates when - their IP address changes. Clients MAY send no certificates at all. - -3. Cell Packet format - - The basic unit of communication for onion routers and onion - proxies is a fixed-width "cell". - - On a version 1 connection, each cell contains the following - fields: - - CircID [2 bytes] - Command [1 byte] - Payload (padded with 0 bytes) [PAYLOAD_LEN bytes] - - On a version 2 connection, all cells are as in version 1 connections, - except for the initial VERSIONS cell, whose format is: - - Circuit [2 octets; set to 0] - Command [1 octet; set to 7 for VERSIONS] - Length [2 octets; big-endian integer] - Payload [Length bytes] - - The CircID field determines which circuit, if any, the cell is - associated with. - - The 'Command' field holds one of the following values: - 0 -- PADDING (Padding) (See Sec 7.2) - 1 -- CREATE (Create a circuit) (See Sec 5.1) - 2 -- CREATED (Acknowledge create) (See Sec 5.1) - 3 -- RELAY (End-to-end data) (See Sec 5.5 and 6) - 4 -- DESTROY (Stop using a circuit) (See Sec 5.4) - 5 -- CREATE_FAST (Create a circuit, no PK) (See Sec 5.1) - 6 -- CREATED_FAST (Circuit created, no PK) (See Sec 5.1) - 7 -- VERSIONS (Negotiate proto version) (See Sec 4) - 8 -- NETINFO (Time and address info) (See Sec 4) - 9 -- RELAY_EARLY (End-to-end data; limited) (See sec 5.6) - - The interpretation of 'Payload' depends on the type of the cell. - PADDING: Payload is unused. - CREATE: Payload contains the handshake challenge. - CREATED: Payload contains the handshake response. - RELAY: Payload contains the relay header and relay body. - DESTROY: Payload contains a reason for closing the circuit. - (see 5.4) - Upon receiving any other value for the command field, an OR must - drop the cell. Since more cell types may be added in the future, ORs - should generally not warn when encountering unrecognized commands. - - The payload is padded with 0 bytes. - - PADDING cells are currently used to implement connection keepalive. - If there is no other traffic, ORs and OPs send one another a PADDING - cell every few minutes. - - CREATE, CREATED, and DESTROY cells are used to manage circuits; - see section 5 below. - - RELAY cells are used to send commands and data along a circuit; see - section 6 below. - - VERSIONS and NETINFO cells are used to set up connections. See section 4 - below. - -4. Negotiating and initializing connections - -4.1. Negotiating versions with VERSIONS cells - - There are multiple instances of the Tor link connection protocol. Any - connection negotiated using the "certificates up front" handshake (see - section 2 above) is "version 1". In any connection where both parties - have behaved as in the "renegotiation" handshake, the link protocol - version is 2 or higher. - - To determine the version, in any connection where the "renegotiation" - handshake was used (that is, where the server sent only one certificate - at first and where the client did not send any certificates until - renegotiation), both parties MUST send a VERSIONS cell immediately after - the renegotiation is finished, before any other cells are sent. Parties - MUST NOT send any other cells on a connection until they have received a - VERSIONS cell. - - The payload in a VERSIONS cell is a series of big-endian two-byte - integers. Both parties MUST select as the link protocol version the - highest number contained both in the VERSIONS cell they sent and in the - versions cell they received. If they have no such version in common, - they cannot communicate and MUST close the connection. - - Since the version 1 link protocol does not use the "renegotiation" - handshake, implementations MUST NOT list version 1 in their VERSIONS - cell. - -4.2. NETINFO cells - - If version 2 or higher is negotiated, each party sends the other a - NETINFO cell. The cell's payload is: - - Timestamp [4 bytes] - Other OR's address [variable] - Number of addresses [1 byte] - This OR's addresses [variable] - - The address format is a type/length/value sequence as given in section - 6.4 below. The timestamp is a big-endian unsigned integer number of - seconds since the unix epoch. - - Implementations MAY use the timestamp value to help decide if their - clocks are skewed. Initiators MAY use "other OR's address" to help - learn which address their connections are originating from, if they do - not know it. Initiators SHOULD use "this OR's address" to make sure - that they have connected to another OR at its canonical address. - - [As of 0.2.0.23-rc, implementations use none of the above values.] - - -5. Circuit management - -5.1. CREATE and CREATED cells - - Users set up circuits incrementally, one hop at a time. To create a - new circuit, OPs send a CREATE cell to the first node, with the - first half of the DH handshake; that node responds with a CREATED - cell with the second half of the DH handshake plus the first 20 bytes - of derivative key data (see section 5.2). To extend a circuit past - the first hop, the OP sends an EXTEND relay cell (see section 5) - which instructs the last node in the circuit to send a CREATE cell - to extend the circuit. - - The payload for a CREATE cell is an 'onion skin', which consists - of the first step of the DH handshake data (also known as g^x). - This value is hybrid-encrypted (see 0.3) to Bob's onion key, giving - an onion-skin of: - PK-encrypted: - Padding [PK_PAD_LEN bytes] - Symmetric key [KEY_LEN bytes] - First part of g^x [PK_ENC_LEN-PK_PAD_LEN-KEY_LEN bytes] - Symmetrically encrypted: - Second part of g^x [DH_LEN-(PK_ENC_LEN-PK_PAD_LEN-KEY_LEN) - bytes] - - The relay payload for an EXTEND relay cell consists of: - Address [4 bytes] - Port [2 bytes] - Onion skin [DH_LEN+KEY_LEN+PK_PAD_LEN bytes] - Identity fingerprint [HASH_LEN bytes] - - The port and address field denote the IPV4 address and port of the next - onion router in the circuit; the public key hash is the hash of the PKCS#1 - ASN1 encoding of the next onion router's identity (signing) key. (See 0.3 - above.) Including this hash allows the extending OR verify that it is - indeed connected to the correct target OR, and prevents certain - man-in-the-middle attacks. - - The payload for a CREATED cell, or the relay payload for an - EXTENDED cell, contains: - DH data (g^y) [DH_LEN bytes] - Derivative key data (KH) [HASH_LEN bytes] - - The CircID for a CREATE cell is an arbitrarily chosen 2-byte integer, - selected by the node (OP or OR) that sends the CREATE cell. To prevent - CircID collisions, when one node sends a CREATE cell to another, it chooses - from only one half of the possible values based on the ORs' public - identity keys: if the sending node has a lower key, it chooses a CircID with - an MSB of 0; otherwise, it chooses a CircID with an MSB of 1. - - (An OP with no public key MAY choose any CircID it wishes, since an OP - never needs to process a CREATE cell.) - - Public keys are compared numerically by modulus. - - As usual with DH, x and y MUST be generated randomly. - -5.1.1. CREATE_FAST/CREATED_FAST cells - - When initializing the first hop of a circuit, the OP has already - established the OR's identity and negotiated a secret key using TLS. - Because of this, it is not always necessary for the OP to perform the - public key operations to create a circuit. In this case, the - OP MAY send a CREATE_FAST cell instead of a CREATE cell for the first - hop only. The OR responds with a CREATED_FAST cell, and the circuit is - created. - - A CREATE_FAST cell contains: - - Key material (X) [HASH_LEN bytes] - - A CREATED_FAST cell contains: - - Key material (Y) [HASH_LEN bytes] - Derivative key data [HASH_LEN bytes] (See 5.2 below) - - The values of X and Y must be generated randomly. - - If an OR sees a circuit created with CREATE_FAST, the OR is sure to be the - first hop of a circuit. ORs SHOULD reject attempts to create streams with - RELAY_BEGIN exiting the circuit at the first hop: letting Tor be used as a - single hop proxy makes exit nodes a more attractive target for compromise. - -5.2. Setting circuit keys - - Once the handshake between the OP and an OR is completed, both can - now calculate g^xy with ordinary DH. Before computing g^xy, both client - and server MUST verify that the received g^x or g^y value is not degenerate; - that is, it must be strictly greater than 1 and strictly less than p-1 - where p is the DH modulus. Implementations MUST NOT complete a handshake - with degenerate keys. Implementations MUST NOT discard other "weak" - g^x values. - - (Discarding degenerate keys is critical for security; if bad keys - are not discarded, an attacker can substitute the server's CREATED - cell's g^y with 0 or 1, thus creating a known g^xy and impersonating - the server. Discarding other keys may allow attacks to learn bits of - the private key.) - - If CREATE or EXTEND is used to extend a circuit, the client and server - base their key material on K0=g^xy, represented as a big-endian unsigned - integer. - - If CREATE_FAST is used, the client and server base their key material on - K0=X|Y. - - From the base key material K0, they compute KEY_LEN*2+HASH_LEN*3 bytes of - derivative key data as - K = H(K0 | [00]) | H(K0 | [01]) | H(K0 | [02]) | ... - - The first HASH_LEN bytes of K form KH; the next HASH_LEN form the forward - digest Df; the next HASH_LEN 41-60 form the backward digest Db; the next - KEY_LEN 61-76 form Kf, and the final KEY_LEN form Kb. Excess bytes from K - are discarded. - - KH is used in the handshake response to demonstrate knowledge of the - computed shared key. Df is used to seed the integrity-checking hash - for the stream of data going from the OP to the OR, and Db seeds the - integrity-checking hash for the data stream from the OR to the OP. Kf - is used to encrypt the stream of data going from the OP to the OR, and - Kb is used to encrypt the stream of data going from the OR to the OP. - -5.3. Creating circuits - - When creating a circuit through the network, the circuit creator - (OP) performs the following steps: - - 1. Choose an onion router as an exit node (R_N), such that the onion - router's exit policy includes at least one pending stream that - needs a circuit (if there are any). - - 2. Choose a chain of (N-1) onion routers - (R_1...R_N-1) to constitute the path, such that no router - appears in the path twice. - - 3. If not already connected to the first router in the chain, - open a new connection to that router. - - 4. Choose a circID not already in use on the connection with the - first router in the chain; send a CREATE cell along the - connection, to be received by the first onion router. - - 5. Wait until a CREATED cell is received; finish the handshake - and extract the forward key Kf_1 and the backward key Kb_1. - - 6. For each subsequent onion router R (R_2 through R_N), extend - the circuit to R. - - To extend the circuit by a single onion router R_M, the OP performs - these steps: - - 1. Create an onion skin, encrypted to R_M's public onion key. - - 2. Send the onion skin in a relay EXTEND cell along - the circuit (see section 5). - - 3. When a relay EXTENDED cell is received, verify KH, and - calculate the shared keys. The circuit is now extended. - - When an onion router receives an EXTEND relay cell, it sends a CREATE - cell to the next onion router, with the enclosed onion skin as its - payload. As special cases, if the extend cell includes a digest of - all zeroes, or asks to extend back to the relay that sent the extend - cell, the circuit will fail and be torn down. The initiating onion - router chooses some circID not yet used on the connection between the - two onion routers. (But see section 5.1. above, concerning choosing - circIDs based on lexicographic order of nicknames.) - - When an onion router receives a CREATE cell, if it already has a - circuit on the given connection with the given circID, it drops the - cell. Otherwise, after receiving the CREATE cell, it completes the - DH handshake, and replies with a CREATED cell. Upon receiving a - CREATED cell, an onion router packs it payload into an EXTENDED relay - cell (see section 5), and sends that cell up the circuit. Upon - receiving the EXTENDED relay cell, the OP can retrieve g^y. - - (As an optimization, OR implementations may delay processing onions - until a break in traffic allows time to do so without harming - network latency too greatly.) - -5.3.1. Canonical connections - - It is possible for an attacker to launch a man-in-the-middle attack - against a connection by telling OR Alice to extend to OR Bob at some - address X controlled by the attacker. The attacker cannot read the - encrypted traffic, but the attacker is now in a position to count all - bytes sent between Alice and Bob (assuming Alice was not already - connected to Bob.) - - To prevent this, when an OR we gets an extend request, it SHOULD use an - existing OR connection if the ID matches, and ANY of the following - conditions hold: - - The IP matches the requested IP. - - The OR knows that the IP of the connection it's using is canonical - because it was listed in the NETINFO cell. - - The OR knows that the IP of the connection it's using is canonical - because it was listed in the server descriptor. - - [This is not implemented in Tor 0.2.0.23-rc.] - -5.4. Tearing down circuits - - Circuits are torn down when an unrecoverable error occurs along - the circuit, or when all streams on a circuit are closed and the - circuit's intended lifetime is over. Circuits may be torn down - either completely or hop-by-hop. - - To tear down a circuit completely, an OR or OP sends a DESTROY - cell to the adjacent nodes on that circuit, using the appropriate - direction's circID. - - Upon receiving an outgoing DESTROY cell, an OR frees resources - associated with the corresponding circuit. If it's not the end of - the circuit, it sends a DESTROY cell for that circuit to the next OR - in the circuit. If the node is the end of the circuit, then it tears - down any associated edge connections (see section 6.1). - - After a DESTROY cell has been processed, an OR ignores all data or - destroy cells for the corresponding circuit. - - To tear down part of a circuit, the OP may send a RELAY_TRUNCATE cell - signaling a given OR (Stream ID zero). That OR sends a DESTROY - cell to the next node in the circuit, and replies to the OP with a - RELAY_TRUNCATED cell. - - When an unrecoverable error occurs along one connection in a - circuit, the nodes on either side of the connection should, if they - are able, act as follows: the node closer to the OP should send a - RELAY_TRUNCATED cell towards the OP; the node farther from the OP - should send a DESTROY cell down the circuit. - - The payload of a RELAY_TRUNCATED or DESTROY cell contains a single octet, - describing why the circuit is being closed or truncated. When sending a - TRUNCATED or DESTROY cell because of another TRUNCATED or DESTROY cell, - the error code should be propagated. The origin of a circuit always sets - this error code to 0, to avoid leaking its version. - - The error codes are: - 0 -- NONE (No reason given.) - 1 -- PROTOCOL (Tor protocol violation.) - 2 -- INTERNAL (Internal error.) - 3 -- REQUESTED (A client sent a TRUNCATE command.) - 4 -- HIBERNATING (Not currently operating; trying to save bandwidth.) - 5 -- RESOURCELIMIT (Out of memory, sockets, or circuit IDs.) - 6 -- CONNECTFAILED (Unable to reach server.) - 7 -- OR_IDENTITY (Connected to server, but its OR identity was not - as expected.) - 8 -- OR_CONN_CLOSED (The OR connection that was carrying this circuit - died.) - 9 -- FINISHED (The circuit has expired for being dirty or old.) - 10 -- TIMEOUT (Circuit construction took too long) - 11 -- DESTROYED (The circuit was destroyed w/o client TRUNCATE) - 12 -- NOSUCHSERVICE (Request for unknown hidden service) - -5.5. Routing relay cells - - When an OR receives a RELAY or RELAY_EARLY cell, it checks the cell's - circID and determines whether it has a corresponding circuit along that - connection. If not, the OR drops the cell. - - Otherwise, if the OR is not at the OP edge of the circuit (that is, - either an 'exit node' or a non-edge node), it de/encrypts the payload - with the stream cipher, as follows: - 'Forward' relay cell (same direction as CREATE): - Use Kf as key; decrypt. - 'Back' relay cell (opposite direction from CREATE): - Use Kb as key; encrypt. - Note that in counter mode, decrypt and encrypt are the same operation. - - The OR then decides whether it recognizes the relay cell, by - inspecting the payload as described in section 6.1 below. If the OR - recognizes the cell, it processes the contents of the relay cell. - Otherwise, it passes the decrypted relay cell along the circuit if - the circuit continues. If the OR at the end of the circuit - encounters an unrecognized relay cell, an error has occurred: the OR - sends a DESTROY cell to tear down the circuit. - - When a relay cell arrives at an OP, the OP decrypts the payload - with the stream cipher as follows: - OP receives data cell: - For I=N...1, - Decrypt with Kb_I. If the payload is recognized (see - section 6..1), then stop and process the payload. - - For more information, see section 6 below. - -5.6. Handling relay_early cells - - A RELAY_EARLY cell is designed to limit the length any circuit can reach. - When an OR receives a RELAY_EARLY cell, and the next node in the circuit - is speaking v2 of the link protocol or later, the OR relays the cell as a - RELAY_EARLY cell. Otherwise, it relays it as a RELAY cell. - - If a node ever receives more than 8 RELAY_EARLY cells on a given - outbound circuit, it SHOULD close the circuit. (For historical reasons, - we don't limit the number of inbound RELAY_EARLY cells; they should - be harmless anyway because clients won't accept extend requests. See - bug 1038.) - - When speaking v2 of the link protocol or later, clients MUST only send - EXTEND cells inside RELAY_EARLY cells. Clients SHOULD send the first ~8 - RELAY cells that are not targeted at the first hop of any circuit as - RELAY_EARLY cells too, in order to partially conceal the circuit length. - - [In a future version of Tor, servers will reject any EXTEND cell not - received in a RELAY_EARLY cell. See proposal 110.] - -6. Application connections and stream management - -6.1. Relay cells - - Within a circuit, the OP and the exit node use the contents of - RELAY packets to tunnel end-to-end commands and TCP connections - ("Streams") across circuits. End-to-end commands can be initiated - by either edge; streams are initiated by the OP. - - The payload of each unencrypted RELAY cell consists of: - Relay command [1 byte] - 'Recognized' [2 bytes] - StreamID [2 bytes] - Digest [4 bytes] - Length [2 bytes] - Data [CELL_LEN-14 bytes] - - The relay commands are: - 1 -- RELAY_BEGIN [forward] - 2 -- RELAY_DATA [forward or backward] - 3 -- RELAY_END [forward or backward] - 4 -- RELAY_CONNECTED [backward] - 5 -- RELAY_SENDME [forward or backward] [sometimes control] - 6 -- RELAY_EXTEND [forward] [control] - 7 -- RELAY_EXTENDED [backward] [control] - 8 -- RELAY_TRUNCATE [forward] [control] - 9 -- RELAY_TRUNCATED [backward] [control] - 10 -- RELAY_DROP [forward or backward] [control] - 11 -- RELAY_RESOLVE [forward] - 12 -- RELAY_RESOLVED [backward] - 13 -- RELAY_BEGIN_DIR [forward] - - 32..40 -- Used for hidden services; see rend-spec.txt. - - Commands labelled as "forward" must only be sent by the originator - of the circuit. Commands labelled as "backward" must only be sent by - other nodes in the circuit back to the originator. Commands marked - as either can be sent either by the originator or other nodes. - - The 'recognized' field in any unencrypted relay payload is always set - to zero; the 'digest' field is computed as the first four bytes of - the running digest of all the bytes that have been destined for - this hop of the circuit or originated from this hop of the circuit, - seeded from Df or Db respectively (obtained in section 5.2 above), - and including this RELAY cell's entire payload (taken with the digest - field set to zero). - - When the 'recognized' field of a RELAY cell is zero, and the digest - is correct, the cell is considered "recognized" for the purposes of - decryption (see section 5.5 above). - - (The digest does not include any bytes from relay cells that do - not start or end at this hop of the circuit. That is, it does not - include forwarded data. Therefore if 'recognized' is zero but the - digest does not match, the running digest at that node should - not be updated, and the cell should be forwarded on.) - - All RELAY cells pertaining to the same tunneled stream have the - same stream ID. StreamIDs are chosen arbitrarily by the OP. RELAY - cells that affect the entire circuit rather than a particular - stream use a StreamID of zero -- they are marked in the table above - as "[control]" style cells. (Sendme cells are marked as "sometimes - control" because they can take include a StreamID or not depending - on their purpose -- see Section 7.) - - The 'Length' field of a relay cell contains the number of bytes in - the relay payload which contain real payload data. The remainder of - the payload is padded with NUL bytes. - - If the RELAY cell is recognized but the relay command is not - understood, the cell must be dropped and ignored. Its contents - still count with respect to the digests, though. - -6.2. Opening streams and transferring data - - To open a new anonymized TCP connection, the OP chooses an open - circuit to an exit that may be able to connect to the destination - address, selects an arbitrary StreamID not yet used on that circuit, - and constructs a RELAY_BEGIN cell with a payload encoding the address - and port of the destination host. The payload format is: - - ADDRESS | ':' | PORT | [00] - - where ADDRESS can be a DNS hostname, or an IPv4 address in - dotted-quad format, or an IPv6 address surrounded by square brackets; - and where PORT is a decimal integer between 1 and 65535, inclusive. - - [What is the [00] for? -NM] - [It's so the payload is easy to parse out with string funcs -RD] - - Upon receiving this cell, the exit node resolves the address as - necessary, and opens a new TCP connection to the target port. If the - address cannot be resolved, or a connection can't be established, the - exit node replies with a RELAY_END cell. (See 6.4 below.) - Otherwise, the exit node replies with a RELAY_CONNECTED cell, whose - payload is in one of the following formats: - The IPv4 address to which the connection was made [4 octets] - A number of seconds (TTL) for which the address may be cached [4 octets] - or - Four zero-valued octets [4 octets] - An address type (6) [1 octet] - The IPv6 address to which the connection was made [16 octets] - A number of seconds (TTL) for which the address may be cached [4 octets] - [XXXX No version of Tor currently generates the IPv6 format.] - - [Tor servers before 0.1.2.0 set the TTL field to a fixed value. Later - versions set the TTL to the last value seen from a DNS server, and expire - their own cached entries after a fixed interval. This prevents certain - attacks.] - - The OP waits for a RELAY_CONNECTED cell before sending any data. - Once a connection has been established, the OP and exit node - package stream data in RELAY_DATA cells, and upon receiving such - cells, echo their contents to the corresponding TCP stream. - RELAY_DATA cells sent to unrecognized streams are dropped. - - Relay RELAY_DROP cells are long-range dummies; upon receiving such - a cell, the OR or OP must drop it. - -6.2.1. Opening a directory stream - - If a Tor server is a directory server, it should respond to a - RELAY_BEGIN_DIR cell as if it had received a BEGIN cell requesting a - connection to its directory port. RELAY_BEGIN_DIR cells ignore exit - policy, since the stream is local to the Tor process. - - If the Tor server is not running a directory service, it should respond - with a REASON_NOTDIRECTORY RELAY_END cell. - - Clients MUST generate an all-zero payload for RELAY_BEGIN_DIR cells, - and servers MUST ignore the payload. - - [RELAY_BEGIN_DIR was not supported before Tor 0.1.2.2-alpha; clients - SHOULD NOT send it to routers running earlier versions of Tor.] - -6.3. Closing streams - - When an anonymized TCP connection is closed, or an edge node - encounters error on any stream, it sends a 'RELAY_END' cell along the - circuit (if possible) and closes the TCP connection immediately. If - an edge node receives a 'RELAY_END' cell for any stream, it closes - the TCP connection completely, and sends nothing more along the - circuit for that stream. - - The payload of a RELAY_END cell begins with a single 'reason' byte to - describe why the stream is closing, plus optional data (depending on - the reason.) The values are: - - 1 -- REASON_MISC (catch-all for unlisted reasons) - 2 -- REASON_RESOLVEFAILED (couldn't look up hostname) - 3 -- REASON_CONNECTREFUSED (remote host refused connection) [*] - 4 -- REASON_EXITPOLICY (OR refuses to connect to host or port) - 5 -- REASON_DESTROY (Circuit is being destroyed) - 6 -- REASON_DONE (Anonymized TCP connection was closed) - 7 -- REASON_TIMEOUT (Connection timed out, or OR timed out - while connecting) - 8 -- (unallocated) [**] - 9 -- REASON_HIBERNATING (OR is temporarily hibernating) - 10 -- REASON_INTERNAL (Internal error at the OR) - 11 -- REASON_RESOURCELIMIT (OR has no resources to fulfill request) - 12 -- REASON_CONNRESET (Connection was unexpectedly reset) - 13 -- REASON_TORPROTOCOL (Sent when closing connection because of - Tor protocol violations.) - 14 -- REASON_NOTDIRECTORY (Client sent RELAY_BEGIN_DIR to a - non-directory server.) - - (With REASON_EXITPOLICY, the 4-byte IPv4 address or 16-byte IPv6 address - forms the optional data, along with a 4-byte TTL; no other reason - currently has extra data.) - - OPs and ORs MUST accept reasons not on the above list, since future - versions of Tor may provide more fine-grained reasons. - - Tors SHOULD NOT send any reason except REASON_MISC for a stream that they - have originated. - - [*] Older versions of Tor also send this reason when connections are - reset. - [**] Due to a bug in versions of Tor through 0095, error reason 8 must - remain allocated until that version is obsolete. - - --- [The rest of this section describes unimplemented functionality.] - - Because TCP connections can be half-open, we follow an equivalent - to TCP's FIN/FIN-ACK/ACK protocol to close streams. - - An exit connection can have a TCP stream in one of three states: - 'OPEN', 'DONE_PACKAGING', and 'DONE_DELIVERING'. For the purposes - of modeling transitions, we treat 'CLOSED' as a fourth state, - although connections in this state are not, in fact, tracked by the - onion router. - - A stream begins in the 'OPEN' state. Upon receiving a 'FIN' from - the corresponding TCP connection, the edge node sends a 'RELAY_FIN' - cell along the circuit and changes its state to 'DONE_PACKAGING'. - Upon receiving a 'RELAY_FIN' cell, an edge node sends a 'FIN' to - the corresponding TCP connection (e.g., by calling - shutdown(SHUT_WR)) and changing its state to 'DONE_DELIVERING'. - - When a stream in already in 'DONE_DELIVERING' receives a 'FIN', it - also sends a 'RELAY_FIN' along the circuit, and changes its state - to 'CLOSED'. When a stream already in 'DONE_PACKAGING' receives a - 'RELAY_FIN' cell, it sends a 'FIN' and changes its state to - 'CLOSED'. - - If an edge node encounters an error on any stream, it sends a - 'RELAY_END' cell (if possible) and closes the stream immediately. - -6.4. Remote hostname lookup - - To find the address associated with a hostname, the OP sends a - RELAY_RESOLVE cell containing the hostname to be resolved with a nul - terminating byte. (For a reverse lookup, the OP sends a RELAY_RESOLVE - cell containing an in-addr.arpa address.) The OR replies with a - RELAY_RESOLVED cell containing a status byte, and any number of - answers. Each answer is of the form: - Type (1 octet) - Length (1 octet) - Value (variable-width) - TTL (4 octets) - "Length" is the length of the Value field. - "Type" is one of: - 0x00 -- Hostname - 0x04 -- IPv4 address - 0x06 -- IPv6 address - 0xF0 -- Error, transient - 0xF1 -- Error, nontransient - - If any answer has a type of 'Error', then no other answer may be given. - - The RELAY_RESOLVE cell must use a nonzero, distinct streamID; the - corresponding RELAY_RESOLVED cell must use the same streamID. No stream - is actually created by the OR when resolving the name. - -7. Flow control - -7.1. Link throttling - - Each client or relay should do appropriate bandwidth throttling to - keep its user happy. - - Communicants rely on TCP's default flow control to push back when they - stop reading. - - The mainline Tor implementation uses token buckets (one for reads, - one for writes) for the rate limiting. - - Since 0.2.0.x, Tor has let the user specify an additional pair of - token buckets for "relayed" traffic, so people can deploy a Tor relay - with strict rate limiting, but also use the same Tor as a client. To - avoid partitioning concerns we combine both classes of traffic over a - given OR connection, and keep track of the last time we read or wrote - a high-priority (non-relayed) cell. If it's been less than N seconds - (currently N=30), we give the whole connection high priority, else we - give the whole connection low priority. We also give low priority - to reads and writes for connections that are serving directory - information. See proposal 111 for details. - -7.2. Link padding - - Link padding can be created by sending PADDING cells along the - connection; relay cells of type "DROP" can be used for long-range - padding. - - Currently nodes are not required to do any sort of link padding or - dummy traffic. Because strong attacks exist even with link padding, - and because link padding greatly increases the bandwidth requirements - for running a node, we plan to leave out link padding until this - tradeoff is better understood. - -7.3. Circuit-level flow control - - To control a circuit's bandwidth usage, each OR keeps track of two - 'windows', consisting of how many RELAY_DATA cells it is allowed to - originate (package for transmission), and how many RELAY_DATA cells - it is willing to consume (receive for local streams). These limits - do not apply to cells that the OR receives from one host and relays - to another. - - Each 'window' value is initially set to 1000 data cells - in each direction (cells that are not data cells do not affect - the window). When an OR is willing to deliver more cells, it sends a - RELAY_SENDME cell towards the OP, with Stream ID zero. When an OR - receives a RELAY_SENDME cell with stream ID zero, it increments its - packaging window. - - Each of these cells increments the corresponding window by 100. - - The OP behaves identically, except that it must track a packaging - window and a delivery window for every OR in the circuit. - - An OR or OP sends cells to increment its delivery window when the - corresponding window value falls under some threshold (900). - - If a packaging window reaches 0, the OR or OP stops reading from - TCP connections for all streams on the corresponding circuit, and - sends no more RELAY_DATA cells until receiving a RELAY_SENDME cell. -[this stuff is badly worded; copy in the tor-design section -RD] - -7.4. Stream-level flow control - - Edge nodes use RELAY_SENDME cells to implement end-to-end flow - control for individual connections across circuits. Similarly to - circuit-level flow control, edge nodes begin with a window of cells - (500) per stream, and increment the window by a fixed value (50) - upon receiving a RELAY_SENDME cell. Edge nodes initiate RELAY_SENDME - cells when both a) the window is <= 450, and b) there are less than - ten cell payloads remaining to be flushed at that edge. - -A.1. Differences between spec and implementation - -- The current specification requires all ORs to have IPv4 addresses, but - allows servers to exit and resolve to IPv6 addresses, and to declare IPv6 - addresses in their exit policies. The current codebase has no IPv6 - support at all. - diff --git a/doc/spec/version-spec.txt b/doc/spec/version-spec.txt deleted file mode 100644 index 842271ae1..000000000 --- a/doc/spec/version-spec.txt +++ /dev/null @@ -1,45 +0,0 @@ -$Id$ - - HOW TOR VERSION NUMBERS WORK - -1. The Old Way - - Before 0.1.0, versions were of the format: - MAJOR.MINOR.MICRO(status(PATCHLEVEL))?(-cvs)? - where MAJOR, MINOR, MICRO, and PATCHLEVEL are numbers, status is one - of "pre" (for an alpha release), "rc" (for a release candidate), or - "." for a release. As a special case, "a.b.c" was equivalent to - "a.b.c.0". We compare the elements in order (major, minor, micro, - status, patchlevel, cvs), with "cvs" preceding non-cvs. - - We would start each development branch with a final version in mind: - say, "0.0.8". Our first pre-release would be "0.0.8pre1", followed by - (for example) "0.0.8pre2-cvs", "0.0.8pre2", "0.0.8pre3-cvs", - "0.0.8rc1", "0.0.8rc2-cvs", and "0.0.8rc2". Finally, we'd release - 0.0.8. The stable CVS branch would then be versioned "0.0.8.1-cvs", - and any eventual bugfix release would be "0.0.8.1". - -2. The New Way - - After 0.1.0, versions are of the format: - MAJOR.MINOR.MICRO(.PATCHLEVEL)(-status_tag) - The stuff in parentheses is optional. As before, MAJOR, MINOR, MICRO, - and PATCHLEVEL are numbers, with an absent number equivalent to 0. - All versions should be distinguishable purely by those four - numbers. The status tag is purely informational, and lets you know how - stable we think the release is: "alpha" is pretty unstable; "rc" is a - release candidate; and no tag at all means that we have a final - release. If the tag ends with "-cvs" or "-dev", you're looking at a - development snapshot that came after a given release. If we *do* - encounter two versions that differ only by status tag, we compare them - lexically. - - Now, we start each development branch with (say) 0.1.1.1-alpha. The - patchlevel increments consistently as the status tag changes, for - example, as in: 0.1.1.2-alpha, 0.1.1.3-alpha, 0.1.1.4-rc, 0.1.1.5-rc. - Eventually, we release 0.1.1.6. The next patch release is 0.1.1.7. - - Between these releases, CVS is versioned with a -cvs tag: after - 0.1.1.1-alpha comes 0.1.1.1-alpha-cvs, and so on. But starting with - 0.1.2.1-alpha-dev, we switched to SVN and started using the "-dev" - suffix instead of the "-cvs" suffix. From ce149c10227bd4d65bfa62898e729430981328c1 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Tue, 22 Feb 2011 12:52:52 -0500 Subject: [PATCH 18/26] That shalt also not have a label without a statement. --- src/common/log.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/common/log.c b/src/common/log.c index 4b21fd916..cfa0721c2 100644 --- a/src/common/log.c +++ b/src/common/log.c @@ -248,6 +248,7 @@ format_msg(char *buf, size_t buf_len, format_msg_no_room_for_domains: /* This will leave end_of_prefix and n unchanged, and thus cause * whatever log domain string we had written to be clobbered. */ + ; } if (funcname && should_log_function_name(domain, severity)) { From cdc59c198a15195a113d68b2c4fbb270792da533 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Tue, 22 Feb 2011 14:06:28 -0500 Subject: [PATCH 19/26] Don't let bad DNS make exit policy and declared exit policy get out of sync Patch from "postman" on trac. Fixes bg 2366. Bug on 0.1.2.5-alpha. --- changes/bug2366 | 8 ++++++++ src/or/policies.c | 8 ++++++++ src/or/policies.h | 1 + src/or/router.c | 15 +++++++++------ 4 files changed, 26 insertions(+), 6 deletions(-) create mode 100644 changes/bug2366 diff --git a/changes/bug2366 b/changes/bug2366 new file mode 100644 index 000000000..d171be453 --- /dev/null +++ b/changes/bug2366 @@ -0,0 +1,8 @@ + o Minor bugfixes + - When a relay decides that its DNS is too broken for it to serve + as an exit server, it advertised itself as a non-exit, but + continued to act as an exit. This could create accidental + partitioning opportunities for users. Instead, if a relay is + going to advertise reject *:* as its exit policy, it should + really act with exit policy "reject *:*". Fixes bug 2366. + Bugfix on Tor 0.1.2.5-alpha. Bugfix by user "postman" on trac. diff --git a/src/or/policies.c b/src/or/policies.c index 62e048cfc..38c2f7c0f 100644 --- a/src/or/policies.c +++ b/src/or/policies.c @@ -858,6 +858,14 @@ policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, return 0; } +/** Add "reject *:*" to the end of the policy in *dest, allocating + * *dest as needed. */ +void +policies_exit_policy_append_reject_star(smartlist_t **dest) +{ + append_exit_policy_string(dest, "reject *:*"); +} + /** Replace the exit policy of r with reject *:*. */ void policies_set_router_exitpolicy_to_reject_all(routerinfo_t *r) diff --git a/src/or/policies.h b/src/or/policies.h index a954ac4f5..b2947c67e 100644 --- a/src/or/policies.h +++ b/src/or/policies.h @@ -41,6 +41,7 @@ addr_policy_result_t compare_addr_to_addr_policy(uint32_t addr, int policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, int rejectprivate, const char *local_address, int add_default_policy); +void policies_exit_policy_append_reject_star(smartlist_t **dest); void policies_set_router_exitpolicy_to_reject_all(routerinfo_t *exitrouter); int exit_policy_is_general_exit(smartlist_t *policy); int policy_is_reject_star(const smartlist_t *policy); diff --git a/src/or/router.c b/src/or/router.c index 59276bac3..4c5eb7a39 100644 --- a/src/or/router.c +++ b/src/or/router.c @@ -1410,9 +1410,14 @@ router_rebuild_descriptor(int force) ri->bandwidthcapacity = hibernating ? 0 : rep_hist_bandwidth_assess(); - policies_parse_exit_policy(options->ExitPolicy, &ri->exit_policy, - options->ExitPolicyRejectPrivate, - ri->address, !options->BridgeRelay); + if (dns_seems_to_be_broken() || has_dns_init_failed()) { + /* DNS is screwed up; don't claim to be an exit. */ + policies_exit_policy_append_reject_star(&ri->exit_policy); + } else { + policies_parse_exit_policy(options->ExitPolicy, &ri->exit_policy, + options->ExitPolicyRejectPrivate, + ri->address, !options->BridgeRelay); + } ri->policy_is_reject_star = policy_is_reject_star(ri->exit_policy); @@ -1866,9 +1871,7 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, } /* Write the exit policy to the end of 's'. */ - if (dns_seems_to_be_broken() || has_dns_init_failed() || - !router->exit_policy || !smartlist_len(router->exit_policy)) { - /* DNS is screwed up; don't claim to be an exit. */ + if (!router->exit_policy || !smartlist_len(router->exit_policy)) { strlcat(s+written, "reject *:*\n", maxlen-written); written += strlen("reject *:*\n"); tmpe = NULL; From 0ab62a4dc531e9f13967aa93bf4dc1925cd69f03 Mon Sep 17 00:00:00 2001 From: Roger Dingledine Date: Tue, 22 Feb 2011 14:39:09 -0500 Subject: [PATCH 20/26] prefer https urls --- doc/spec/README | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/spec/README b/doc/spec/README index a7fa17002..ccd33a421 100644 --- a/doc/spec/README +++ b/doc/spec/README @@ -1,10 +1,11 @@ The Tor specifications and proposals have moved to a new repository. -To browse the specifications, go to +To browse the specifications, go to https://gitweb.torproject.org/torspec.git/tree To check out the specification repository, run git clone git://git.torproject.org/torspec.git For other information on the repository, see - http://gitweb.torproject.org/torspec.git + https://gitweb.torproject.org/torspec.git + From 372773d681ed0f67e824cf2eef41a7186ab6c458 Mon Sep 17 00:00:00 2001 From: Roger Dingledine Date: Tue, 22 Feb 2011 14:50:00 -0500 Subject: [PATCH 21/26] changes file for removing torspec from the tarball --- changes/torspec.git | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 changes/torspec.git diff --git a/changes/torspec.git b/changes/torspec.git new file mode 100644 index 000000000..ba33ca20f --- /dev/null +++ b/changes/torspec.git @@ -0,0 +1,5 @@ + o Packaging changes: + - Stop shipping the Tor specs files and development proposal documents + in the tarball. They are now in a separate git repository at + git://git.torproject.org/torspec.git + From e606f19fa369c7ad5934793cb4f72f97972b52bc Mon Sep 17 00:00:00 2001 From: Roger Dingledine Date: Tue, 22 Feb 2011 14:50:00 -0500 Subject: [PATCH 22/26] changes file for removing torspec from the tarball --- changes/torspec.git | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 changes/torspec.git diff --git a/changes/torspec.git b/changes/torspec.git new file mode 100644 index 000000000..ba33ca20f --- /dev/null +++ b/changes/torspec.git @@ -0,0 +1,5 @@ + o Packaging changes: + - Stop shipping the Tor specs files and development proposal documents + in the tarball. They are now in a separate git repository at + git://git.torproject.org/torspec.git + From 10ad3442e11fb5a54c19eef7425bc317a9cc0969 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Tue, 22 Feb 2011 17:51:03 -0500 Subject: [PATCH 23/26] Remove doc/spec/Makefile.in from list of generated files --- configure.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure.in b/configure.in index 68c593a11..7cdb330b4 100644 --- a/configure.in +++ b/configure.in @@ -866,7 +866,7 @@ fi CPPFLAGS="$CPPFLAGS $TOR_CPPFLAGS_libevent $TOR_CPPFLAGS_openssl $TOR_CPPFLAGS_zlib" -AC_CONFIG_FILES([Makefile tor.spec Doxyfile contrib/tor.sh contrib/torctl contrib/torify contrib/tor.logrotate contrib/Makefile contrib/osx/Makefile contrib/osx/TorBundleDesc.plist contrib/osx/TorBundleInfo.plist contrib/osx/TorDesc.plist contrib/osx/TorInfo.plist contrib/osx/TorStartupDesc.plist src/config/torrc.sample doc/tor.1 src/Makefile doc/Makefile doc/design-paper/Makefile doc/spec/Makefile src/config/Makefile src/common/Makefile src/or/Makefile src/win32/Makefile src/tools/Makefile contrib/suse/Makefile contrib/suse/tor.sh]) +AC_CONFIG_FILES([Makefile tor.spec Doxyfile contrib/tor.sh contrib/torctl contrib/torify contrib/tor.logrotate contrib/Makefile contrib/osx/Makefile contrib/osx/TorBundleDesc.plist contrib/osx/TorBundleInfo.plist contrib/osx/TorDesc.plist contrib/osx/TorInfo.plist contrib/osx/TorStartupDesc.plist src/config/torrc.sample doc/tor.1 src/Makefile doc/Makefile doc/design-paper/Makefile src/config/Makefile src/common/Makefile src/or/Makefile src/win32/Makefile src/tools/Makefile contrib/suse/Makefile contrib/suse/tor.sh]) AC_OUTPUT if test -x /usr/bin/perl && test -x ./contrib/updateVersions.pl ; then From a2727f62494b8a56152214017ac8fb348dd064be Mon Sep 17 00:00:00 2001 From: Roger Dingledine Date: Tue, 22 Feb 2011 17:54:25 -0500 Subject: [PATCH 24/26] prevent same entry and exit for insane edge case --- changes/bug2403 | 6 ++++++ src/or/circuitbuild.c | 3 ++- 2 files changed, 8 insertions(+), 1 deletion(-) create mode 100644 changes/bug2403 diff --git a/changes/bug2403 b/changes/bug2403 new file mode 100644 index 000000000..3b29b37fb --- /dev/null +++ b/changes/bug2403 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - In the special case where you configure a public exit relay as your + bridge, Tor would be willing to use that exit relay as the last + hop in your circuit as well. Now we fail that circuit instead. + Bugfix on 0.2.0.12-alpha. Fixes bug 2403. Reported by "piebeer". + diff --git a/src/or/circuitbuild.c b/src/or/circuitbuild.c index 4f8f5fbab..b3c9f0e1b 100644 --- a/src/or/circuitbuild.c +++ b/src/or/circuitbuild.c @@ -4063,7 +4063,6 @@ choose_random_entry(cpath_build_state_t *state) int preferred_min, consider_exit_family = 0; if (chosen_exit) { - smartlist_add(exit_family, chosen_exit); routerlist_add_family(exit_family, chosen_exit); consider_exit_family = 1; } @@ -4086,6 +4085,8 @@ choose_random_entry(cpath_build_state_t *state) r = entry_is_live(entry, need_uptime, need_capacity, 0, &msg); if (!r) continue; /* down, no point */ + if (r == chosen_exit) + continue; /* don't pick the same node for entry and exit */ if (consider_exit_family && smartlist_isin(exit_family, r)) continue; /* avoid relays that are family members of our exit */ if (options->EntryNodes && From 530e87ce316f8a3feb1b116c89cc703090217e2d Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Tue, 22 Feb 2011 18:12:46 -0500 Subject: [PATCH 25/26] Add some dollar signs in an attempt to appease older asciidocs We should really require a modern asiidoc: backporting stuff to 8.2 is a timesink. --- doc/tor.1.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 033f0a28e..8b8fd202b 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -330,7 +330,7 @@ Other options can be specified either on the command-line (--option set of "logging domains". Each logging domain corresponds to an area of functionality inside Tor. You can specify any number of severity ranges for a single log statement, each of them prefixed by a comma-separated - list of logging domains. You can prefix a domain with ~ to indicate + list of logging domains. You can prefix a domain with $$~$$ to indicate negation, and use * to indicate "all domains". If you specify a severity range without a list of domains, it matches all domains. + + From 0642b927928df4ec556a76fed4b0b4523d515a7d Mon Sep 17 00:00:00 2001 From: Karsten Loesing Date: Thu, 24 Feb 2011 16:44:54 +0100 Subject: [PATCH 26/26] Fix two potential bugs in the bug1863 code. --- src/or/rephist.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/or/rephist.c b/src/or/rephist.c index 265fff8a3..3f4a70418 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -1513,7 +1513,7 @@ rep_hist_update_bwhist_state_section(or_state_t *state, smartlist_free(*s_values); } if (*s_maxima) { - SMARTLIST_FOREACH(*s_values, char *, val, tor_free(val)); + SMARTLIST_FOREACH(*s_maxima, char *, val, tor_free(val)); smartlist_free(*s_maxima); } if (! server_mode(get_options())) { @@ -1541,7 +1541,7 @@ rep_hist_update_bwhist_state_section(or_state_t *state, i = (b->num_maxes_set <= b->next_max_idx) ? 0 : b->next_max_idx; for (j=0; j < b->num_maxes_set; ++j,++i) { uint64_t maxval; - if (i > NUM_TOTALS) + if (i >= NUM_TOTALS) i = 0; tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(b->totals[i] & ~0x3ff)); smartlist_add(*s_values, cp);