Small Updates + Clarifications

src/SUMMARY.md

@@ -1,14 +1,16 @@
 # Summary
 
-- [Overview](./overview.md)
+- [Problem Overview](./overview.md)
 - [Risk Model](./risk.md)
+- [Cwtch Overview](./cwtch-overview.md)
 - [Open Questions](./open-questions.md)
-- [Connectivity](./connectivity.md)
-- [Tapir](./tapir.md)
-    - [Packet Format](./packet_format.md)md
-    - [Authentication Protocol](./authentication_protocol.md)
 - [Cwtch Library](./cwtch.md)
+    - [Message Formats](./message_formats.md)
     - [Groups](./groups.md)
+    - [Tapir](./tapir.md)
+      - [Packet Format](./packet_format.md)
+      - [Authentication Protocol](./authentication_protocol.md)
+    - [Connectivity](./connectivity.md)
 - [Cwtch UI](./ui.md)
     - [Profile Encryption & Storage](./profile_encryption_and_storage.md)
     - [Android Service](./android.md)

src/cwtch-overview.md

@@ -0,0 +1,45 @@
+# Cwtch Overview
+
+This page presents a brief technical overview of the Cwtch system.
+
+
+## A Cwtch Profile
+
+Users can create one of more Cwtch Profiles. Each profile generates a random ed25519 keypair compatible with
+ Tor.
+
+In addition to the cryptographic material, a profile also contains a list of Contacts (other Cwtch profile public keys +
+associated data about that profile like nickname and (optionally) historical messages), a list of Groups (containing the group cryptographic material in
+addition to other associated data like the group nickname and historical messages).
+
+## 2-party conversions: Peer to Peer
+
+![](./BASE_3.png)
+
+For 2 parties to engage in a peer-to-peer conversation both must be online, but only one needs to be reachable via
+their onion service. For the sake of clarity we often label one party the "inbound peer" (the one who hosts the onion service) and the other party the
+"outbound peer" (the one that connects to the onion service).
+
+After connection both parties engage in an authentication protocol which:
+
+* Asserts that each party has access to the private key associated with their public identity.
+* Generates an ephemeral session key used to encrypt all further communication during the session.
+
+This exchange (documented in further detail in [authentication protocol](./authentication_protocol.md)) is *offline deniable*
+i.e. it is possible for any party to forge transcripts of this protocol exchange after the fact, and as such - after the
+fact - it is impossible to definitely prove that the exchange happened at all.
+
+After, the authentication protocol the two parties may exchange messages with each other freely.
+
+## Multi-party conversations: Peer to Server
+
+**Note: Metadata Resistant Group Communication is still an active research area and what is documented here
+will likely change in the future.**
+
+In many respects communication with a server is identical to communication with a regular Cwtch peer,
+all the same steps above are taken however the server always acts as the inbound peer, and the outbound
+peer always uses newly generated keypair as their "longterm identity".
+
+As such peer-server conversations only differ in the *kinds* of messages that are sent between the two parties,
+with the server relaying all messages that it receives and also allowing any client to query for older messages.
+

src/cwtch.md

@@ -37,8 +37,7 @@ version of the protobufs framework.
 We have removed protobufs entirely from the project by porting this
 functionality over the Tapir.
 
-<!-- PoW as in Proof of Work? -->
-### PoW Spam Prevention as a Metadata Vector
+### Proof of Work (PoW) Spam Prevention as a Metadata Vector
 
 **Status: Outdated**: Cwtch now uses Token Based Services to separate challenges like PoW from
 resolving the tokens. 

src/groups.md

@@ -58,5 +58,3 @@ chain (which if they were attempting to censor other messages would
  
 Finally: We are actively working on adding non-repudiation to Cwtch servers such
 that they themselves are restricted in what they can censor efficiently.
-
-<!-- I might have missed it but I think this is the first mention of each message being signed by the sender (and that the message contains unique identifiers)? Might be great to have a 'message_format.md' where message structure/constructure is explained/introduced earlier? I also can't tell how the session key is applied to the message at this point.  -->

src/input.md

@@ -3,7 +3,7 @@
 
 ## Risk: Interception of Cwtch content or metadata through an IME on Mobile Devices
 
-**Status: Unmitigated**
+**Status: Partially Mitigated**
 
 Any component that has the potential to intercept data between a person, and the Cwtch app is a 
 potential security risk.
@@ -19,4 +19,7 @@ ecosystem.
 
 A similar risk exists on desktop through the use of similar input applications (in addition to software keyloggers),
 however we consider that fully outside the scope of Cwtch risk assessment (in line with other attacks on the security of the underlying 
-operating system itself).
+operating system itself).
+
+This is partially mitigated in Cwtch 1.2 through the use of `enableIMEPersonalizedLearning: false`. See
+[this PR](https://git.openprivacy.ca/cwtch.im/cwtch-ui/pulls/142) for more information.

src/message_formats.md

@@ -0,0 +1,57 @@
+# Message Formats
+
+## Peer to Peer Messages
+
+    PeerMessage {
+        ID      string // A unique Message ID (primarily used for acknowledgments)
+        Context string // A unique context identifier i.e. im.cwtch.chat
+        Data    []byte // The context-dependent serialized data packet.
+    }
+
+### Context Identifiers
+
+* `im.cwtch.raw` - Data contains a plain text chat message (see: [overlays](./overlays.md) for more information)
+* `im.cwtch.acknowledgement` - Data is empty and ID references a previously sent message
+  
+* `im.cwtch.getVal` and `im.cwtch.retVal` - Used for requesting / returning specific information about a peer. Data
+contains a serialized `peerGetVal` structure and `peerRetVal` respectively.
+    
+        peerGetVal struct {
+                Scope string
+                Path string
+        }
+
+        type peerRetVal struct {
+            Val    string // Serialized path-dependent value
+            Exists bool
+        }
+
+## Plaintext / Decrypted Group Messages
+
+    type DecryptedGroupMessage struct {
+        Text      string // plaintext of the message
+        Onion     string // The cwtch address of the sender
+        Timestamp uint64 // A user specified timestamp
+        // NOTE: SignedGroupID is now a misnomer, the only way this is signed is indirectly via the signed encrypted group messages
+        // We now treat GroupID as binding to a server/key rather than an "owner" - additional validation logic (to e.g.
+        // respect particular group constitutions) can be built on top of group messages, but the underlying groups are
+        // now agnostic to those models.
+        SignedGroupID      []byte 
+        PreviousMessageSig []byte // A reference to a previous message
+        Padding            []byte // random bytes of length = 1800 - len(Text)
+    }
+
+DecryptedGroupMessage contains random padding to a fixed size that is equal to the length of all fixed length fields + 1800.
+This ensures that all encrypted group messages are equal length.
+
+## Encrypted Group Messages
+    
+    // EncryptedGroupMessage provides an encapsulation of the encrypted group message stored on the server
+    type EncryptedGroupMessage struct {
+        Ciphertext []byte
+        Signature  []byte // Sign(groupID + group.GroupServer + base64(decrypted group message)) using the senders cwtch key
+    }
+
+Calculating the signature requires knowing the groupID of the message, the server the group is associated with
+and the decrypted group message (and thus, the Group Key). It is (ed25519) signed by the sender of the message, and can be
+verified using their public cwtch address key.

src/overlays.md

@@ -1,7 +1,10 @@
 # Message Overlays
-<!-- Oh, great. Can this be earlier in the handbook? Like section 2. early? -->
+
 [Adapted from: Discreet Log #8: Notes on the Cwtch Chat API](https://openprivacy.ca/discreet-log/08-chatapi/)
 
+**Note: This section covers overlay protocols on-top of the Cwtch protcol. For information on the Cwtch Protocol
+messages themselves please see [Message Formats](./message_formats.md)**
+
 We envision Cwtch as a platform for providing an authenticated transport layer to higher-level applications. 
 Developers are free to make their own choices about what application layer protocols to use, 
 whether they want bespoke binary message formats or just want to throw an HTTP library on top and call it a 
@@ -65,7 +68,6 @@ The most simple over is a chat message which simply contains raw, unprocessed ch
 ```
 {o:1,d:"got milk?"}
 ```
-<!-- I was told there would unique identifiers and signatures? -->
 
 ## Invitations (Overlays 100 and 101)
 
@@ -80,4 +82,6 @@ The wire format for these are equally simple:
 
 This represents a departure from our original "overlays" thinking to a more action-oriented representation. The chat "overlay" can communicate that someone *did* something, even if it's paraphrased down to "added an item to a list," and the lists and bulletins and other beautifully chaotic data can have their state precomputed and stored separately. 
 
-<!-- What are lists or bulletin boards? -->
+## Lists / Bulletin Boards
+
+**Note: Expected to be Defined in Cwtch Beta 1.5**

src/profile_encryption_and_storage.md

@@ -10,8 +10,6 @@ it isn't possible to provide a full list of profiles a user might have access to
 To handle profiles that are "unencrypted" (i.e don't require a password to open) we currently create a profile
 with a [defacto, hardcoded password](https://git.openprivacy.ca/cwtch.im/libcwtch-go/src/branch/trunk/constants/globals.go#L5).
 
-<!-- Can cwtch be updated to randomly generate a password and store it on the devices keychain/keystore (for devices/OS with a keychain API)? Also I'm not sure how secure OS level keychains are but hard coded passwords always become campfire stories years later. -->
-
 This isn't ideal, we would much rather wish to rely on OS-provided key material such that the profile is bound to a 
 specific device, but such features are currently patchwork - we also note by creating an unencrypted profile, people
 who use Cwtch are explicitly opting into the risk that someone with access to the file system may be able to decrypt

src/server.md

@@ -1,5 +1,5 @@
 # Cwtch Server
-<!-- Can this be moved up to be section 3. of the handbook? -->
+
 The goal of the Cwtch protocol is to enable group communication through
 **Untrusted Infrastructure**.
 

src/tapir.md

@@ -100,25 +100,18 @@ we expect it to be protected via a preceeding app in an `ApplicationChain` e.g.
 
 ### Ephemeral Connections
 
-Occasionally it is desirable to have a peer connect to another / a service
+Occasionally it is desirable to have a peer connect to a service
 without using their long term identity (e.g. in the case of connecting to
 a Cwtch Server).
 
 In this case we want to enable a convenient way to allow connecting with an
-ephemeral identity. 
+**ephemeral identity**.
 
-<!-- This is confusing "avoid side channel around avoid
-duplicate connections" -->
 It turns out that doing this securely requires maintaining a completely separate
-set of connections and applications in order to avoid side channel around avoid
-duplicate connections (i.e. if we did mix them up then a service might be able
-to exploit the fact that clients avoid duplicate connections by attempting to
-connect to known-online peers and observing if they reject the connection
- because they already have an outbound ephemeral connection open.)
- 
-Because of this, we don't provide an explicit Ephemeral Connect api and instead
-recommend that peers maintain one long term service and multiple ephemeral
- services. 
+set of connections and applications in order to avoid side channels caused by duplicate connections handling.
+
+As such the Cwtch Protocol Engine maintains two disctinct connection pools, one for avowed connections and another
+for ephemeral connections. All connections to known Cwtch Servers are made through the ephemeral pool.
 
 ## Known Risks  
 

src/ui.md

@@ -58,8 +58,6 @@ In order to be suitable to deploy groups at a wide scale, the app requires a way
 to prevent Cwtch from fetching information over such connections, and this
 should likely be turned on by default.
 
-<!-- Can messages from peers/servers be received/sent in a batch? -->
-
 
 ## Testing Status