diff options
Diffstat (limited to 'protocols/Tox/toxcore/docs')
| -rw-r--r-- | protocols/Tox/toxcore/docs/Group-Chats.md | 71 | ||||
| -rw-r--r-- | protocols/Tox/toxcore/docs/Hardening.txt | 60 | ||||
| -rw-r--r-- | protocols/Tox/toxcore/docs/Hardening_docs.txt | 30 | ||||
| -rw-r--r-- | protocols/Tox/toxcore/docs/Prevent_Tracking.txt | 158 | ||||
| -rw-r--r-- | protocols/Tox/toxcore/docs/TCP_Network.txt | 154 | ||||
| -rw-r--r-- | protocols/Tox/toxcore/docs/TODO | 62 | ||||
| -rw-r--r-- | protocols/Tox/toxcore/docs/Tox_middle_level_network_protocol.txt | 120 | ||||
| -rw-r--r-- | protocols/Tox/toxcore/docs/av_api.md | 194 | ||||
| -rw-r--r-- | protocols/Tox/toxcore/docs/updates/Crypto.md | 54 | ||||
| -rw-r--r-- | protocols/Tox/toxcore/docs/updates/DHT.md | 105 | ||||
| -rw-r--r-- | protocols/Tox/toxcore/docs/updates/Spam-Prevention.md | 12 | ||||
| -rw-r--r-- | protocols/Tox/toxcore/docs/updates/Symmetric-NAT-Transversal.md | 43 |
12 files changed, 0 insertions, 1063 deletions
diff --git a/protocols/Tox/toxcore/docs/Group-Chats.md b/protocols/Tox/toxcore/docs/Group-Chats.md deleted file mode 100644 index 5e00b550cc..0000000000 --- a/protocols/Tox/toxcore/docs/Group-Chats.md +++ /dev/null @@ -1,71 +0,0 @@ -Massive public group chats. - -Note that not all this document has been implemented: only private (invite only) group chats are currently implemented. - -Everyone generates a short term public private key pair right before joining -the chat. - -Note that for public group chats it is impossible to protect the chat from -being spied on by a very dedicated attacker, encryption is therefor used as a -form of spam/access control. - -## Joining the chats - - -## Protocol - - -Node format: -See DHT, currently uses the IPv6 Node_format. - -Get nodes (Request): -Packet contents: -``` -[char with a value of 48][Bob's (The receiver's) Public key (client_id) (32 bytes))][Alice's (The sender's) Public key (client_id) (32 bytes)][Random nonce (24 bytes)][Encrypted with the nonce, private key of the sender and public key of the receiver:[char with a value of 48][random 8 byte (ping_id)] -``` -Valid replies: a send_nodes packet - -Send_nodes (response): -``` -[char with a value of 48][Bob's (The receiver's) Public key (client_id) (32 bytes))][Alice's (The sender's) Public key (client_id) (32 bytes)][Random nonce (24 bytes)][Encrypted with the nonce, private key of the sender and public key of the receiver:[char with a value of 49][random 8 byte (ping_id)][Nodes in node format, length=40 * (number of nodes (maximum of 6 nodes)) bytes]] -``` - -Broadcast packet: -``` -[char with a value of 48][Bob's (The receiver's) Public key (client_id) (32 bytes))][Alice's (The sender's) Public key (client_id) (32 bytes)][nonce][Encrypted with the nonce, private key of the sender and public key of the receiver:[char with a value of 50][Data to send to everyone]] -``` - - -Data to send to everyone: -TODO: signing and spam control + permissions. -[client_id of sender][uint32_t message number][char with a value representing id of message][data] - -Note: the message number is increased by 1 for each sent message. - -message ids: -0 - ping -sent every ~60 seconds by every peer. -No data. - -16 - new_peer -Tell everyone about a new peer in the chat. -[uint8_t public_key[public_key_len]] - -17 - ban_peer -Ban a peer -[uint8_t public_key[public_key_len]] - -18 - topic change -[uint8_t topic[topiclen]] - -48 - name change -[uint8_t name[namelen]] - -49 - status change -[uint8_t (status id)] - -64 - chat message -[uint8_t message[messagelen]] - -65 - action (/me) -[uint8_t message[messagelen]] diff --git a/protocols/Tox/toxcore/docs/Hardening.txt b/protocols/Tox/toxcore/docs/Hardening.txt deleted file mode 100644 index 50ccfd53e0..0000000000 --- a/protocols/Tox/toxcore/docs/Hardening.txt +++ /dev/null @@ -1,60 +0,0 @@ -Currently an attacker with sufficient resources could launch a large scale -denial of service type attack by flooding the Tox network with a bunch of nodes -that do not act like real nodes to prevent people from finding each other. - -Due to the design of Tox, this is the worst thing an attacker can do to disrupt -the network. - -This solution's goal is to make these denial of service attack very very hard -to accomplish. - -For the network to work every Tox node must: -1. Respond to ping requests. -2. Respond to get node requests with the ids of nodes closest to a queried id -(It is assumed each nodes know at least the 32 nodes closest to them.) -3. Properly send crypto request packets to their intended destination. - -Currently the only thing a node needs to do to be part of the network is -respond correctly to ping requests. - -The only people we really trust on the network are the nodes in our friends -list. - - -The behavior of each Tox node is easily predictable. This means that it possible -for Tox nodes to test the nodes that they are connected to to see if they -behave like normal Tox nodes and only send nodes that are confirmed to behave -like real Tox nodes as part of send node replies when other nodes query them. - -If correctly done, this means that to poison the network an attacker can only -infiltrate the network if his "fake" nodes behave exactly like real nodes -completely defeating the purpose of the attack. Of course nodes must be -rechecked regularly to defeat an attack where someone floods the network with -many good nodes then suddenly turns them all bad. - -This also prevents someone from accidentally killing the tox network with a bad -implementation of the protocol. - -Implementation ideas (In Progress): - -1. Use our friends to check if the nodes in our close list are good. - -EX: If our friend queries a node close to us and it correctly returns our -ip/port and then sends a crypto request packet to it and it routes it correctly -to us then it is good. - -Problems with this: People don't always have at least one online friend. - -2. Pick random nodes (add ourselves some random (fake) friends to increase the -pool of available nodes) and make then send requests to other nodes, the -response is then relayed back to us and compared to how the node should have -behaved. If the node is found to be behaving correctly, it is set as trusted. -Only trusted nodes are sent in send node packets, that is unless the exact node -being queried for in the getnode packet is present, it will be sent in the -sendnode packet even if it is not trusted. - -The hypothesis is that if to be part of the network nodes have to behave -correctly it should prevent disruption from nodes that behave incorrectly. - -(This idea is currently being implemented in the harden branch.) -... diff --git a/protocols/Tox/toxcore/docs/Hardening_docs.txt b/protocols/Tox/toxcore/docs/Hardening_docs.txt deleted file mode 100644 index 53a6103418..0000000000 --- a/protocols/Tox/toxcore/docs/Hardening_docs.txt +++ /dev/null @@ -1,30 +0,0 @@ -Hardening request packets are sent as crypto request packets (see crypto docs.) -NOTE: currently only get nodes requests are tested in the code which is why -there is only one test (more will be added soon.) - -All hardening requests must contain exactly 768 bytes of data. (The data sent -must be padded with zeros if it is smaller than that.) - -1. Get the information (IP_port, client_id) of the node we want to test. -2. Find a couple random nodes that is not that node (one for each test.) -3. Send crypto request packets to each of these random nodes with the data being: - -[byte with value: 02 (get nodes test request)][struct Node_format (the node to -test.)][client_id(32 bytes) the id to query the node with.][padding] - -4. The random node receives a packet. --The packet is a get nodes test request: - send a get_node request to that node with the id to query in the request. - when a send_node response is received, send the following response to the - person who sent us the get nodes test request packet: - [byte with value: 03 (get nodes test response)][client_id(32 bytes): - the id of the tested node][The list of nodes it responded with in IPv6 - Node format (struct Node_Format)] - PROTIP: (get node requests and response contain an encrypted part that you - can use to store information so that you don't - have to store in your memory where/if to send back the response from the - send node) - -5. Receive the test responses. --If the test(s) pass (the nodes behave in a satisfactory manner), make these -nodes have priority over those who don't pass the test(s). diff --git a/protocols/Tox/toxcore/docs/Prevent_Tracking.txt b/protocols/Tox/toxcore/docs/Prevent_Tracking.txt deleted file mode 100644 index d170103f0b..0000000000 --- a/protocols/Tox/toxcore/docs/Prevent_Tracking.txt +++ /dev/null @@ -1,158 +0,0 @@ -Current privacy issues with the Tox DHT: - -1. It makes tracking people across different IPs very easy. -Solution: Have each new DHT use a temporary public/private key pair not related -to the long term public/private key pair. - -2. Metadata on which key is friends to which can be collected (The hardening -makes this somewhat harder by introducing a bunch of random traffic but not -impossible.). -Solution: If no long term keys were used in the DHT it would solve this -problem. (possibly knowing which ip is connected to which is much less -precious.) - - -So, it seems all our privacy problems are solved if we can manage to make every -node in the DHT have a keypair that is not related to the long term keys and is -generated every session. - - -So, every node in the DHT now has a temporary keypair not related to their real -long term one. - -But, how do people find themselves then? We have to add a way for people to -tell their friends what their DHT public key is. We also have to somehow make -it so people can send/receive friend requests. This has to be done without -non-friends being able to find out where a node is. - -The solution: Onion routing + enable the storage of some small amount of data -on DHT nodes. - - -Alice and bob are friends. Before joining the DHT they generate temporary -session keypairs to be used for the DHT instead of their long term keys. - -Bob finds a bunch of random nodes then picks 3 random working ones (A, B, C). - -Bob gets the known working node with an id closest to his real one from his list (D) - -Bob then creates an onion (the packet will go through A, B, C and will end up at D) -announce request packet with his real public key, ping_id as zeros and -searching for his real public key. - -Bob will announce response packets and will recursively send onion announce request -packets to closer and closer nodes until he finds the ones closest to his real public key. - -Once he has done this, he will send some onion announce request packets with the right -ping_id previously received from the node when he queried it to announce himself to the node. - -The nodes he announces himself to keep the information to send onion packets to that node in -memory. - -Alice meanwhile searches for the nodes closest to Bobs real id using a temporary keypair and -announce request packets. She does this until she finds nodes that respond with a ping_id of zero. - -She sends data to route request packet with information telling Bob her temporary id in the DHT -(or a friend request if she is not friends with him). - -Bob finds her by using her temporary id and they connect to each other. - - -NOTE: crypto_box is used for all the asymmetric encryption and crypto_secretbox is used for all -the symmetric. Also every DHT node have a random symmetric key which they use to encrypt the stuff -in normal get node request that is used to encrypt stuff in the following. - -Onion packet (request): - -initial (sent from us to node A): - -[uint8_t packet id (128)][nonce] -[our temp DHT public key]encrypted with our temp DHT private key and the pub key of Node A and the nonce:[ -[IP_Port of node B][a random public key]encrypted with the random private key and the pub key of Node B and the nonce:[ -[IP_Port of node C][a random public key]encrypted with the random private key and the pub key of Node C and the nonce:[ -[IP_Port of node D][data to send to Node D]]]] - -(sent from node A to node B): - -[uint8_t packet id (129)][nonce] -[a random public key]encrypted with the random private key and the pub key of Node B and the nonce:[ -[IP_Port of node C][a random public key]encrypted with the random private key and the pub key of Node C and the nonce:[ -[IP_Port of node D][data to send to Node D]]][nonce (for the following symmetric encryption)]encrypted with temp symmetric key of Node A: [IP_Port (of us)] - -(sent from node B to node C): - -[uint8_t packet id (130)][nonce] -[a random public key]encrypted with the random private key and the pub key of Node C and the nonce:[ -[IP_Port of node D][data to send to Node D]][nonce (for the following symmetric encryption)] -encrypted with temp symmetric key of Node B:[IP_Port (of Node A)[nonce (for the following symmetric encryption)] -encrypted with temp symmetric key of Node A: [IP_Port (of us)]] - -(sent from node C to node D): -[data to send to Node D][nonce (for the following symmetric encryption)]encrypted with temp symmetric key of Node C: -[IP_Port (of Node B)[nonce (for the following symmetric encryption)] -encrypted with temp symmetric key of Node B:[IP_Port (of Node A)[nonce (for the following symmetric encryption)] -encrypted with temp symmetric key of Node A: [IP_Port (of us)]]] - -Data sent to Node D: - -announce request packet: -[uint8_t packet id (131)][nonce][our real long term public key or a temporary one (see next)] -encrypted (with our real long term private key if we want to announce ourselves, a temporary one if we are searching for friends) and the pub key of Node D and the nonce: -[[(32 bytes) ping_id][client id we are searching for][public key that we want those sending back data packets to use.][data to send back in response(fixed size)]] - -(if the ping id is zero, respond with a announce response packet) -(If the ping id matches the one the node sent in the announce response and the public key matches the one being searched for, -add the part used to send data to our list (if the list is full make it replace the furthest entry)) - -data to route request packet: -[uint8_t packet id (133)][public key of destination node][nonce][temporary just generated public key] -encrypted with that temporary private key and the nonce and the public key from the announce response packet of the destination node:[data] -(if Node D contains the ret data for the node, it sends the stuff in this packet as a data to route response packet to the right node) - -The data in the previous packet is in format: [real public key of sender] -encrypted with real private key of the sender, the nonce in the data packet and -the real public key of the receiver:[[uint8_t id][data (optional)]] - -Data sent to us: -announce response packet: -[uint8_t packet id (132)][data to send back in response(fixed size)][nonce] -encrypted with the DHT private key of Node D, the public key in the request and the nonce:[[uint8_t is_stored] -[(32 bytes) ping_id if is_stored is 0, public key that must be used to send data packets if is_stored is not 0][Node_Format * (maximum of 8)]] -(if the is_stored is not 0, it means the information to reach the client id we are searching for is stored on this node) - -data to route response packet: -[uint8_t packet id (134)][nonce][temporary just generated public key] -encrypted with that temporary private key, the nonce and the public key from the announce response packet of the destination node:[data] - - -Onion packet (response): - -initial (sent from node D to node C): - -[uint8_t packet id (140)][nonce (for the following symmetric encryption)]encrypted with temp symmetric key of Node C: -[IP_Port (of Node B)[nonce (for the following symmetric encryption)] -encrypted with temp symmetric key of Node B:[IP_Port (of Node A)[nonce (for the following symmetric encryption)] -encrypted with temp symmetric key of Node A: [IP_Port (of us)]]][data to send back] - -(sent from node C to node B): - -[uint8_t packet id (141)][nonce (for the following symmetric encryption)] -encrypted with temp symmetric key of Node B:[IP_Port (of Node A)[nonce (for the following symmetric encryption)] -encrypted with temp symmetric key of Node A: [IP_Port (of us)]][data to send back] - -(sent from node B to node A): - -[uint8_t packet id (142)][nonce (for the following symmetric encryption)] -encrypted with temp symmetric key of Node A: [IP_Port (of us)][data to send back] - -(sent from node A to us): - -[data to send back] - - -Data packets: - -To tell our friend what our DHT public key is so that he can connect to us we send a data packet -with id 156 and the data being:[uint64_t (in network byte order) no_replay, the packet will only be -accepted if this number is bigger than the last one received] [our dht public key][Node_Format * ( -maximum of 8) nodes closest to us so that the friend can find us faster] diff --git a/protocols/Tox/toxcore/docs/TCP_Network.txt b/protocols/Tox/toxcore/docs/TCP_Network.txt deleted file mode 100644 index 4bc2d433ac..0000000000 --- a/protocols/Tox/toxcore/docs/TCP_Network.txt +++ /dev/null @@ -1,154 +0,0 @@ -It has come to our attention that to achieve decent market penetration Tox -must work behind ALL internet connections, may they be behind enterprise NATs -or any other bad network conditions. - -The people who have issues with the UDP direct connection approach seem to be a -small minority though it is hard to estimate how many. - -This means that routing their packets using good nodes on the network will -probably not take a huge toll on the network and will assure that people -can use Tox regardless of the quality of their internet connection. - - -How it's going to work: -1. Alice, a Tox client on a TCP only network generates a temporary public key -and connects to a bootstrap node. - -2. Using the bootstrap node she finds and connects to a couple (exact number -to be determined later) number of random nodes that have TCP relay support. - -3. She uses the onion through the TCP relay connections to send friend requests -or tell online friends which TCP nodes she is connected to and her temporary -public key. - -4. Bob receives an onion packet from Alice telling him which nodes she is -connected to. Bob connects to these nodes and establishes a routed connection -with alice using that temporary public key. - -5. That connection is used by both to transmit encrypted Messenger and A/V -packets. - -6. If one of the nodes shuts down while it is currently routing traffic, Alice -and bob just switch to one of the other nodes they are both connected to. - - -Detailed implementation details: - -There are two distinct parts for TCP relays, the client part and the server -part. - -The server acts as the actual relay. Servers must have fully forwarded TCP -ports (NAT-PMP and uPNP can help here). The first port the server will try -binding to is 443 followed by port 3389 and possibly some others. Onion packets -can be sent/received through the TCP servers. - - -Server: - -The public/private key pair the TCP server uses is the same one he uses for the -DHT. - -all crypto for communication with the server uses the crypto_box() function of -NaCl. - -TCP doesn't have packets so what we will refer to as packets are sent this way: -[[uint16_t (length of data)][data]] - -So if you would inspect the TCP stream you would see: -[[uint16_t (length of data)][data]][[uint16_t (length of -data)][data]][[uint16_t (length of data)][data]] - -Note that both handshake packets don't have this format (the length for them is -always the same so we don't need to specify it.) - -When the client connects to the server, he sends this packet: -[public key of client (32 bytes)][nonce for the encrypted data [24 -bytes]][encrypted with the private key of the client and public key of the -server and the nonce:[public key (32 bytes) and][base nonce we want the server -to use to encrypt the packets sent to us (24 bytes)]] - -The server responds with: -[nonce for the encrypted data [24 bytes]][encrypted with the public key of the -client and private key of the server and the nonce:[public key (32 bytes) -and][base nonce we want the client to use to encrypt the packets sent to us (24 -bytes)]] - -All packets to the server are end to end encrypted with the information -received -(and sent) in the handshake. - -(first packet is encrypted with the base nonce the private key for which the -client sent the server the public key and the public key we sent to the client, -the next with base nonce + 1...) - -The connection is set to an unconfirmed state until a packet is received and -decrypted correctly using the information in the handshake. - -each packet sent to/from the server has an id (the first byte of the plain text -data of the packet.) - -ids 0 to 15 are reserved for special packets, ids 16 to 255 are used to denote -who we want the data to be routed to/who the packet is from. - -special ids and packets: -0 - Routing request. -[uint8_t id (0)][public key (32 bytes)] -1 - Routing request response. -[uint8_t id (1)][uint8_t (rpid) 0 if refused, packet id if accepted][public key -(32 bytes)] -2 - Connect notification: -[uint8_t id (2)][uint8_t (packet id of connection that got connected)] -3 - Disconnect notification: -[uint8_t id (3)][uint8_t (packet id of connection that got disconnected)] -4 - ping packet -[uint8_t id (4)][uint64_t ping_id (0 is invalid)] -5 - ping response (pong) -[uint8_t id (5)][uint64_t ping_id (0 is invalid)] -6 - OOB send -[uint8_t id (6)][destination public key (32 bytes)][data] -7 - OOB recv -[uint8_t id (7)][senders public key (32 bytes)][data] -8 - onion packet (same format as initial onion packet (See: Prevent -tracking.txt) but packet id is 8 instead of 128) -9 - onion packet response (same format as onion packet with id 142 but id is 9 -instead.) - -The rest of the special ids are reserved for possible future usage. - -If the server receives a routing request he stores server side that the client -wants to connect to the person with that public key and sends back a Routing -request response with the rpid along with the public key sent in the request. - -If for some reason the server must refuse the routing request (too many) he -sends the response with a rpid of 0. - -If the person who the client wants to connect to is also online and wants to -connect to the client a connect notification is sent to both with the -appropriate packet id. - -If either one disconnects, a disconnect notification is sent to the other with -appropriate packet id. - -If a client sends a disconnect notification, the entry on the server for that -routed connection is cleared and a disconnect notification is sent to the peer -(if he was online) - -If the server receives an onion packet he handles it the same as he would if it -was one received normally via UDP, he must also assure himself that any -responses must be sent to the proper client. - -Ping responses must have the same ping_id as the request. - -If the server receives a ping packet he must respond with a ping response. - -The server will send a ping packet to clients every 30 seconds, they have 30 -seconds to respond, if they don't the connection is deleted. - -OOB send packets will be sent to the peer connected to the TCP server with the -destination public key as a OOB recv packet. The client sending this packet has -no way of knowing if the packet reached its destination. - - -Client: - -Implementation details coming soon. diff --git a/protocols/Tox/toxcore/docs/TODO b/protocols/Tox/toxcore/docs/TODO deleted file mode 100644 index dea513b25d..0000000000 --- a/protocols/Tox/toxcore/docs/TODO +++ /dev/null @@ -1,62 +0,0 @@ -TODO list. - -[IN PROGRESS] Add what is left to do to the TODO list. - -Networking: - [NOT STARTED] UPnP port forwarding. - [NOT STARTED] NAT-PMP port forwarding. - -DHT: - [ALMOST DONE] Metadata collection prevention. (docs/Prevent_Tracking.txt) - [IN PROGRESS] Hardening against attacks. - [IN PROGRESS] Optimizing the code. - -Lossless UDP: - [DONE] Increase data send/receive rates. - -[IN PROGRESS] Massive IRC like group chats (text only) - [DONE] Networking base. - [NOT STARTED] Syncing chat state between clients (nicknames, list of who is in chat, etc...) - [NOT STARTED] Make clients sign their messages so that peers can't modify them. - -[IN PROGRESS] Audio/Video - [DONE] encoding/streaming/decoding - [DONE] Call initiation - [DONE] Encryption - [IN PROGRESS] Auditing. - [NEEDS TESTING] Video packet splitting. - [IN PROGRESS] Prevent audio skew (seems to be easily solvable client side.) - [IN PROGRESS] Group chats. - -Friend_requests.c: - [NOT STARTED] What happens when a friend request is received needs to be changed. - [NOT STARTED] Add multiple nospam functionality. - -[DONE] File transfers -[NOT STARTED] Offline messaging -[NOT STARTED] Friends list syncing -[IN PROGRESS] IPV6 support - [DONE] Networking - [DONE] DHT + Messenger - [NOT STARTED] Group chats (They work with IPv6 but some things need to be tested.) - - -[IN PROGRESS] Make toxcore thread safe. - -[NOT STARTED] Make the core save/datafile portable across client versions/different processor architectures. - -[MOSTLY DONE] A way for people to connect to people on Tox if they are behind a bad NAT that -blocks UDP (or is just unpunchable) (docs/TCP_Network.txt) - -[NEEDS TESTING] Make the save made with tox_save_encrypted(...) harder to brute force. -See: (https://github.com/jencka/ProjectTox-libtoxdata) - -[IN PROGRESS] GUI (no official one chosen yet, a list of promising ones follows) -https://github.com/notsecure/uTox -https://github.com/naxuroqa/Venom -https://github.com/Impyy/Toxy -https://github.com/lehitoskin/blight -https://github.com/nurupo/ProjectTox-Qt-GUI -https://github.com/Astonex/Antox - -[NOT STARTED] Security audit from professionals diff --git a/protocols/Tox/toxcore/docs/Tox_middle_level_network_protocol.txt b/protocols/Tox/toxcore/docs/Tox_middle_level_network_protocol.txt deleted file mode 100644 index a84132b6ca..0000000000 --- a/protocols/Tox/toxcore/docs/Tox_middle_level_network_protocol.txt +++ /dev/null @@ -1,120 +0,0 @@ -The TCP client and TCP server part are in a state that can be considered -feature complete. Why doesn't Tox support TCP yet even if those parts are -complete? - -The answer is that a way to ensure a smooth switchover between the TCP and UDP -needs to be added. If Tox first connects to the other user using TCP but then -due to pure chance manages to connect using the faster direct UDP connection -Tox must switch seamlessly from the TCP to the UDP connection without there -being any data loss or the other user going offline and then back online. The -transition must be seamless whatever both connected users are doing be it -transferring files or simply chatting together. - -Possible evil/bad or simply TCP relays going offline must not impact the -connection between both clients. - -Typically Tox will use more than one TCP relay to connect to other peers for -maximum connection stability which means there must be a way for Tox to take -advantage of multiple relays in a way that the user will never be aware if one -of them goes offline/tries to slow down the connection/decides to corrupt -packets/etc.. - -To accomplish this Tox needs something between the low level protocol (TCP) and -high level Tox messaging protocol hence the name middle level. - -The plan is to move some functionality from lossless_UDP to a higher level: -more specifically the functionality for detecting which packets a peer is -missing and the ability to request and send them again. lossless UDP uses plain -text packets to request missing packets from the other peer while Tox is -currently designed to kill the connection if any packet tampering is detected. -This works very well when connecting directly with someone because if the -attacker can modify packets it means he can kill your connection anyways. With -TCP relays however that is not the case as such the packets used to request -missing packets must be encrypted. If it is detected that a packet has been -tampered, the connection must stay intact while the evil relay must be -disconnected from and replaced with a good relay, the behavior must be the same -as if the relay had just suddenly gone online. Of course something to protect -from evil "friends" framing relays must also be implemented. - -Detailed implementation details: - -cookie request packet: -[uint8_t 24][Senders DHT Public key (32 bytes)][Random nonce (24 -bytes)][Encrypted message containing: [Senders real public key (32 -bytes)][padding (32 bytes)][uint64_t number (must be sent -back untouched in cookie response)]] -Encrypted message is encrypted with sender DHT private key, recievers DHT -public key and the nonce. - -cookie response packet: -[uint8_t 25][Random nonce (24 bytes)][Encrypted message containing: -[Cookie][uint64_t number (that was sent in the request)]] -Encrypted message is encrypted with sender DHT private key, recievers DHT -public key and the nonce. - -The Cookie should be basically: -[nonce][encrypted data:[uint64_t time][Senders real public key (32 -bytes)][Senders dht public key (32 bytes)]] - -Handshake packet: -[uint8_t 26][Cookie][nonce][Encrypted message containing: [random 24 bytes base -nonce][session public key of the peer (32 bytes)][sha512 hash of the entire -Cookie sitting outside the encrypted part][Other Cookie (used by the other to -respond to the handshake packet)]] - -The handshake packet is encrypted using the real private key of the sender, the -real private key of the receiver and the nonce. - - -Alice wants to connect to bob. - -Alice sends a cookie request packet to bob and gets a cookie response back. - -Alice then generates a nonce and a temporary public/private keypair. - -Alice then takes that nonce and just generated private key, the obtained -cookie, creates a new cookie and puts them in a handshake packet which she -sends to bob. - -Bob gets the handshake packet, accepts the connection request, then generates a -nonce and a temporary public/private keypair and sends a handshake packet back -with this just generated information and with the cookie field being the Other -Cookie contained in the received handshake. - -Both then use these temporary keys to generate the session key with which every -data packet sent and received will be encrypted and decrypted. The nonce sent -in the handshake will be used to encrypt the first data packet sent, the nonce -+ 1 the second, the nonce + 2 the third and so on. - -Data packets: - -[uint8_t 27][uint16_t (in network byte order) the last 2 bytes of the nonce -used to encrypt this][encrypted with the session key and a nonce:[plain data]] - -Plain data in the data packets: - -[uint32_t our recvbuffers buffer_start, (highest packet number handled + -1)][uint32_t packet number if lossless, our sendbuffer buffer_end if -lossy][data] - -data ids: -0: padding (skipped until we hit a non zero (data id) byte) -1: packet request packet (lossy packet) -2: connection kill packet (lossy packet) (tells the other that the connection is over) -... -16+: reserved for Messenger usage (lossless packets). -192+: reserved for Messenger usage (lossy packets). -255: reserved for Messenger usage (lossless packet) - -packet request packet: [uint8_t (1)][uint8_t num][uint8_t num][uint8_t -num]...[uint8_t num] - -the list of nums are a list of packet numbers the other is requesting. -to get the real packet numbers from this list take the recvbuffers buffer_start -from the packet, substract 1 to it and put it in packet_num then start from the -beggining of the num list: if num is zero, add 255 to packet_num then do the -next num. if num isn't zero, add its value to packet_num, note that the other -has requested we send this packet again to them then continue to the next num in -the list. - - diff --git a/protocols/Tox/toxcore/docs/av_api.md b/protocols/Tox/toxcore/docs/av_api.md deleted file mode 100644 index 2f536ade8b..0000000000 --- a/protocols/Tox/toxcore/docs/av_api.md +++ /dev/null @@ -1,194 +0,0 @@ -#A/V API reference - -##Take toxmsi/phone.c as a reference - -###Initialization: - -``` -phone_t* initPhone(uint16_t _listen_port, uint16_t _send_port); -``` - -function initializes sample phone. _listen_port and _send_port are variables only meant -for local testing. You will not have to do anything regarding to that since -everything will be started within a mesenger. - - -Phone requires one msi session and two rtp sessions ( one for audio and one for -video ). - -``` -msi_session_t* msi_init_session( void* _core_handler, const uint8_t* _user_agent ); -``` - -initializes msi session. -Params: - -``` -void* _core_handler - pointer to an object handling networking, -const uint8_t* _user_agent - string describing phone client version. -``` - -Return value: -msi_session_t* - pointer to a newly created msi session handler. - -###msi_session_t reference: - -How to handle msi session: -Controlling is done via callbacks and action handlers. -First register callbacks for every state/action received and make sure -NOT TO PLACE SOMETHING LIKE LOOPS THAT TAKES A LOT OF TIME TO EXECUTE; every callback is being called -directly from event loop. You can find examples in phone.c. - -Register callbacks: -``` -void msi_register_callback_call_started ( MCALLBACK ); -void msi_register_callback_call_canceled ( MCALLBACK ); -void msi_register_callback_call_rejected ( MCALLBACK ); -void msi_register_callback_call_ended ( MCALLBACK ); - -void msi_register_callback_recv_invite ( MCALLBACK ); -void msi_register_callback_recv_ringing ( MCALLBACK ); -void msi_register_callback_recv_starting ( MCALLBACK ); -void msi_register_callback_recv_ending ( MCALLBACK ); -void msi_register_callback_recv_error ( MCALLBACK ); - -void msi_register_callback_requ_timeout ( MCALLBACK ); -``` - -MCALLBACK is defined as: void (*callback) (void* _arg) -msi_session_t* handler is being thrown as \_arg so you can use that and \_agent_handler to get to your own phone handler -directly from callback. - - -Actions: - -``` -int msi_invite ( msi_session_t* _session, call_type _call_type, uint32_t _timeoutms ); -``` - -Sends call invite. Before calling/sending invite msi_session_t::_friend_id is needed to be set or else -it will not work. _call_type is type of the call ( Audio/Video ) and _timeoutms is how long -will poll wait until request is terminated. - -``` -int msi_hangup ( msi_session_t* _session ); -``` -Hangs up active call - -``` -int msi_answer ( msi_session_t* _session, call_type _call_type ); -``` -Answer incomming call. _call_type set's callee call type. - -``` -int msi_cancel ( msi_session_t* _session ); -``` -Cancel current request. - -``` -int msi_reject ( msi_session_t* _session ); -``` -Reject incomming call. - - -###Now for rtp: - -You will need 2 sessions; one for audio one for video. -You start them with: -``` -rtp_session_t* rtp_init_session ( int _max_users, int _multi_session ); -``` - -Params: -``` -int _max_users - max users. -1 if undefined -int _multi_session - any positive number means uses multi session; -1 if not. -``` - -Return value: -``` -rtp_session_t* - pointer to a newly created rtp session handler. -``` - -###How to handle rtp session: -Take a look at -``` -void* phone_handle_media_transport_poll ( void* _hmtc_args_p ) in phone.c -``` -on example. Basically what you do is just receive a message via: -``` -struct rtp_msg_s* rtp_recv_msg ( rtp_session_t* _session ); -``` - -and then you use payload within the rtp_msg_s struct. Don't forget to deallocate it with: -void rtp_free_msg ( rtp_session_t* _session, struct rtp_msg_s* _msg ); -Receiving should be thread safe so don't worry about that. - -When you capture and encode a payload you want to send it ( obviously ). - -first create a new message with: -``` -struct rtp_msg_s* rtp_msg_new ( rtp_session_t* _session, const uint8_t* _data, uint32_t _length ); -``` - -and then send it with: -``` -int rtp_send_msg ( rtp_session_t* _session, struct rtp_msg_s* _msg, void* _core_handler ); -``` - -_core_handler is the same network handler as in msi_session_s struct. - - -##A/V initialization: -``` -int init_receive_audio(codec_state *cs); -int init_receive_video(codec_state *cs); -Initialises the A/V decoders. On failure it will print the reason and return 0. On success it will return 1. - -int init_send_audio(codec_state *cs); -int init_send_video(codec_state *cs); -Initialises the A/V encoders. On failure it will print the reason and return 0. On success it will return 1. -init_send_audio will also let the user select an input device. init_send_video will determine the webcam's output codec and initialise the appropriate decoder. - -int video_encoder_refresh(codec_state *cs, int bps); -Reinitialises the video encoder with a new bitrate. ffmpeg does not expose the needed VP8 feature to change the bitrate on the fly, so this serves as a workaround. -In the future, VP8 should be used directly and ffmpeg should be dropped from the dependencies. -The variable bps is the required bitrate in bits per second. -``` - - -###A/V encoding/decoding: -``` -void *encode_video_thread(void *arg); -``` -Spawns the video encoding thread. The argument should hold a pointer to a codec_state. -This function should only be called if video encoding is supported (when init_send_video returns 1). -Each video frame gets encoded into a packet, which is sent via RTP. Every 60 frames a new bidirectional interframe is encoded. -``` -void *encode_audio_thread(void *arg); -``` -Spawns the audio encoding thread. The argument should hold a pointer to a codec_state. -This function should only be called if audio encoding is supported (when init_send_audio returns 1). -Audio frames are read from the selected audio capture device during intitialisation. This audio capturing can be rerouted to a different device on the fly. -Each audio frame is encoded into a packet, and sent via RTP. All audio frames have the same amount of samples, which is defined in AV_codec.h. -``` -int video_decoder_refresh(codec_state *cs, int width, int height); -``` -Sets the SDL window dimensions and creates a pixel buffer with the requested size. It also creates a scaling context, which will be used to convert the input image format to YUV420P. - -``` -void *decode_video_thread(void *arg); -``` -Spawns a video decoding thread. The argument should hold a pointer to a codec_state. The codec_state is assumed to contain a successfully initialised video decoder. -This function reads video packets and feeds them to the video decoder. If the video frame's resolution has changed, video_decoder_refresh() is called. Afterwards, the frame is displayed on the SDL window. -``` -void *decode_audio_thread(void *arg); -``` -Spawns an audio decoding thread. The argument should hold a pointer to a codec_state. The codec_state is assumed to contain a successfully initialised audio decoder. -All received audio packets are pushed into a jitter buffer and are reordered. If there is a missing packet, or a packet has arrived too late, it is treated as a lost packet and the audio decoder is informed of the packet loss. The audio decoder will then try to reconstruct the lost packet, based on information from previous packets. -Audio is played on the default OpenAL output device. - - -If you have any more qustions/bug reports/feature request contact the following users on the irc channel #tox-dev on irc.freenode.net: -For RTP and MSI: mannol -For audio and video: Martijnvdc diff --git a/protocols/Tox/toxcore/docs/updates/Crypto.md b/protocols/Tox/toxcore/docs/updates/Crypto.md deleted file mode 100644 index cee03f0aa9..0000000000 --- a/protocols/Tox/toxcore/docs/updates/Crypto.md +++ /dev/null @@ -1,54 +0,0 @@ -Encryption library used: http://nacl.cr.yp.to/ - - -When running the program for the first time the crypto_box_keypair() function is used to -generate the users public-private key pair. (32 bytes each) - -The generated public key is set as the client_id of the peer. - -Adding a friend ---------------- - -Alice adds bob to her friends list by adding his 32 byte public key (client_id) to her friends list. -2 cases: -case 1: Alice adds Bobs public key and bob waits for Alice to attempt to connect to him. -case 2: Bob and Alice add their respective public keys to their friends list at the same time. - -case 1: -Alice sends a onion data (see: Prevent_tracking.txt) packet to bob with the encrypted part containing the friends request like so: -``` -[char with a value of 32][nospam number (4 bytes)][Message] -``` - -Ex message: hello bob it's me alice -_- add me pl0x. - -For more info on the nospam see: Spam_Prevention.txt - -Bob receives the request and decrypts the message using the function crypto_box_open() - -If the message decrypts successfully: -If Alice is already in Bobs friends list: case 2 -If Alice is not in Bob's friends list and the nospam is good: Bob is prompt to add Alice and is shown the message from her. -If Bobs accepts Alice's friends request he adds her public key to his friends list. - -case 2: -Bob and Alice both have the others public key in their friends list, they are ready for the next step: Connecting to an already added friend - -In the next step only crypto_box() is used for encryption and only crypto_box_open() for decryption (just like in the last step.) - - -Connecting to an already added friend -------------------------------------- - -see: Tox_middle_level_network_protocol.txt - -Crypto request packets --------------------------------------- - -``` -[char with a value of 32][Bob's (The reciever's) Public key (client_id) (32 bytes))][Alice's (The sender's) Public key (client_id) (32 bytes)][Random nonce (24 bytes)][Encrypted message] -``` - -The encrypted message is encrypted with crypto_box() (using Bobs public key, Alice's private key and the nonce (randomly generated 24 bytes)) and is a message from Alice in which she tells Bob who she is. - -Each node can route the request to the receiver if they are connected to him. This is to bypass bad NATs. diff --git a/protocols/Tox/toxcore/docs/updates/DHT.md b/protocols/Tox/toxcore/docs/updates/DHT.md deleted file mode 100644 index 17db70cecd..0000000000 --- a/protocols/Tox/toxcore/docs/updates/DHT.md +++ /dev/null @@ -1,105 +0,0 @@ -DHT protocol -============ - -Follows pretty much the principle of the torrent DHT: http://www.bittorrent.org/beps/bep_0005.html (READ IT) - -But: -Vastly simplified packet format and encryption. - -Boostrapping: -The first time you install the client we bootstrap it with a node. (bandwidth should not be a problem as the client only needs to be sent one reply.) - - -Basics ------- -(All the numbers here are just guesses and are probably not optimal values) - -client list: A list of node ids closest (mathematically see bittorrent doc) to ours matched with ip addresses + port number corresponding to that id and a timestamp containing the time or time since the client was successfully pinged. - -"friends" list: A list containing the node_ids of all our "friends" or clients we want to connect to. -Also contains the ip addresses + port + node_ids + timestamp(of last ping like in the client list) of the 8 clients closest (mathematically see bittorrent doc) to each "friend" - -One pinged lists: --One for storing a list of ips along with their ping_ids and a timestamp for the ping requests -Entries in the pinged lists expire after 5 seconds. -If one of the lists becomes full, the expire rate reduces itself one second or the new ping takes the place of the oldest one. - - -Entries in client list and "friends" list expire after 300 seconds without ping response. -Each client stores a maximum of 32 entries in its client list. -Each client in the client list and "friends" list is pinged every 60 seconds. -Each client in the client list and "friends" list has a timestamp which denote the last time it was successfully pinged. -If the corresponding clients timestamp is more than 130 seconds old it is considered bad. -Send a get nodes request every 20 seconds to a random good node for each "friend" in our "friends" list. -Send a get nodes request every 20 seconds to a random good node in the client list. - - -When a client receives any request from another ------------------------------------------------ --Respond to the request - -Ping request is replied to with with a ping response containing the same encrypted data - -Get nodes request is replied with a send nodes reply containing the same encrypted data and the good nodes from the client list and/or the "friends" list that are closest to the requested_node_id - --If the requesting client is not in the client list: - -If there are no bad clients in the list and the list is full: - -If the id of the other client is closer (mathematically see bittorrent doc) than at least one of the clients in the list or our "friends" list: - -Send a ping request to the client. - -if not forget about the client. - - -If there are bad clients and/or the list isn't full: - -Send a ping request to the client - -When a client receives a response ---------------------------------- --Ping response - -If the node was previously pinged with a matching ping_id (check in the corresponding pinged list.) - -If the node is in the client list the matching client's timestamp is set to current time. - -If the node is in the "friends" list the matching client's timestamp is set to current time for every occurrence. - -If the node is not in the client list: - -If the list isn't full, add it to the list. - -If the list is full, the furthest away (mathematically see bittorrent doc) bad client is replaced by the new one. - -If the list is filled with good nodes replace the furthest client with it only if it is closer than the replaced node. - -for each friend in the "friends" list: - -If that friend's client list isn't full, add that client to it - -If that friend's client list contains bad clients, replace the furthest one with that client. - -If that friend's client list contains only good clients - -If the client is closer to the friend than one of the other clients, it replaces the farthest one - -If not, nothing happens. - - -Send nodes - -If the ping_id matches what we sent previously (check in the corresponding pinged list.): - -Each node in the response is pinged. - - - - - -Protocol --------- - -Node format: -``` -[uint8_t family (2 == IPv4, 10 == IPv6, 130 == TCP IPv4, 138 == TCP IPv6)][ip (in network byte order), length=4 bytes if ipv4, 16 bytes if ipv6][port (in network byte order), length=2 bytes][char array (node_id), length=32 bytes] -``` -see also: DHT.h (pack_nodes() and unpack_nodes()) - -Valid queries and Responses: - -Ping(Request and response): -``` -[byte with value: 00 for request, 01 for response][char array (client node_id), length=32 bytes][random 24 byte nonce][Encrypted with the nonce and private key of the sender: [1 byte type (0 for request, 1 for response)][random 8 byte (ping_id)]] -``` -ping_id = a random integer, the response must contain the exact same number as the request - - -Get nodes (Request): -Packet contents: -``` -[byte with value: 02][char array (client node_id), length=32 bytes][random 24 byte nonce][Encrypted with the nonce and private key of the sender:[char array: requested_node_id (node_id of which we want the ip), length=32 bytes][Sendback data (must be sent back unmodified by in the response), length=1 to NODES_ENCRYPTED_MESSAGE_LENGTH bytes]] -``` -Valid replies: a send_nodes packet - -Send_nodes (response (for all addresses)): -``` -[byte with value: 04][char array (client node_id), length=32 bytes][random 24 byte nonce][Encrypted with the nonce and private key of the sender:[uint8_t number of nodes in this packet][Nodes in node format, length=?? * (number of nodes (maximum of 8 nodes)) bytes][Sendback data, length=1 to NODES_ENCRYPTED_MESSAGE_LENGTH bytes]] -``` diff --git a/protocols/Tox/toxcore/docs/updates/Spam-Prevention.md b/protocols/Tox/toxcore/docs/updates/Spam-Prevention.md deleted file mode 100644 index a0713fd1e6..0000000000 --- a/protocols/Tox/toxcore/docs/updates/Spam-Prevention.md +++ /dev/null @@ -1,12 +0,0 @@ - -Situation 1: -Someone randomly goes around the DHT sending friend requests to everyone. - -Prevented by: -Every friend address: -[client_id (32 bytes)][nospam number (4 bytes)][checksum (2 bytes)] -contains a number (nospam). - -The nospam in every friend request to that friend must be that number. - -If not it is rejected. diff --git a/protocols/Tox/toxcore/docs/updates/Symmetric-NAT-Transversal.md b/protocols/Tox/toxcore/docs/updates/Symmetric-NAT-Transversal.md deleted file mode 100644 index 490382160f..0000000000 --- a/protocols/Tox/toxcore/docs/updates/Symmetric-NAT-Transversal.md +++ /dev/null @@ -1,43 +0,0 @@ -Notes: - -Friend requests need to be routed. - -The current DHT should be capable of punching all NATs except symmetric ones. - -###### - -Symmetric NAT hole punching: - -If we are not connected to the friend and if the DHT is queried and ips -returned for the friend are the same but the port is different, the friend is -assumed to be behind a symmetric NAT. - -Before attempting the procedure we first send a routed ping request to the -friend. This request is to be routed through the nodes who returned the ip of -the peer. - -As soon as we receive one routed ping request from the other peer, we respond -with a ping response. - -Ping request/response packet: -See: Crypto request packets in [[Crypto]] - -Message: -For the ping request: -[char with a value of 254][char with 0][8 byte random number] - -For the ping response: -[char with a value of 254][char with 1][8 byte random number (The same that was sent in the request)] - -As soon as we get a proper ping response from the other we run the different -ports returned by the DHT through our port guessing algorithm. - -###### - -Port guessing algorithm: - -Right now it just tries all the ports directly beside the known ports.(A better one is needed) - -###### - -We send DHT ping requests to all the guessed ports, only a couple at a time. |