More updates

Author: tannewt

README.rst

@@ -90,6 +90,15 @@ The service has two characteristics:
 * version (``0x0100``) - Simple unsigned 32-bit integer version number. May be 1 or 2.
 * raw transfer (``0x0200``) - Bidirectional link with a custom protocol. The client does WRITE_NO_RESPONSE to the characteristic and then server replies via NOTIFY. (This is similar to the Nordic UART Service but on a single characteristic rather than two.) The commands over the transfer characteristic are idempotent and stateless. A disconnect during a command will reset the state.
 
+Time resolution
+---------------
+
+Time resolution varies based filesystem type. FATFS can only get down to the 2 second bound after 1980. Littlefs can do 64-bit nanoseconds after January 1st, 1970.
+
+To account for this, the protocol has time in 64-bit nanoseconds after January 1st, 1970. However, the server will respond with a potentially truncated version that is the value stored.
+
+Also note that devices serving the file transfer protocol may not have it's own clock so do not rely on time ordering. Any internal writes may set the time incorrectly. So, we only recommend using the value as a cache key.
+
 Commands
 ---------
 
@@ -150,7 +159,7 @@ The header is four fixed entries and a variable length path:
 * Path length: 16-bit number encoding the encoded length of the path string.
 * Offset: 32-bit number encoding the starting offset to write.
 * Total size: 32-bit number encoding the total length of the file contents.
-* Current time: 64-bit number encoding nanoseconds since January 1st, 1970. Used as the file modification time. Not all system will s
+* Current time: 64-bit number encoding nanoseconds since January 1st, 1970. Used as the file modification time. Not all system will support the full resolution. Use the truncated time response value for caching.
 * Path: UTF-8 encoded string that is *not* null terminated. (We send the length instead.)
 
 The server will repeatedly respond until the total length has been transferred with:
@@ -159,6 +168,7 @@ The server will repeatedly respond until the total length has been transferred w
 * 2 Bytes reserved for padding.
 * Offset: 32-bit number encoding the starting offset to write. (Should match the offset from the previous 0x20 or 0x22 message)
 * Free space: 32-bit number encoding the amount of data the client can send.
+* Truncated time: 64-bit number encoding nanoseconds since January 1st, 1970 as stored by the file system. The resolution may be less that the protocol. It is sent back for use in caching on the host side.
 
 The client will repeatedly respond until the total length has been transferred with:
 * Command: Single byte. Always ``0x22``.
@@ -170,16 +180,7 @@ The client will repeatedly respond until the total length has been transferred w
 
 The transaction is complete after the server has received all data and replied with a status with 0 free space and offset set to the content length.
 
-**NOTE**: Current time was added in version 2. The rest of the packets remained the same.
-
-Time resolution
-~~~~~~~~~~~~~~~
-
-Time resolution varies based filesystem type. FATFS can only get down to the 2 second bound after 1980. Littlefs can do 64-bit nanoseconds after Jan 1, 1970.
-
-What do you want to do?
-* We could either return the stored time when doing a write.
-* We could add file system info command that has the smallest resolution in units of 64-bit nanoseconds. This command could to total filesystem size and free space as well.
+**NOTE**: Current time was added in version 3. The rest of the packets remained the same.
 
 
 ``0x30`` - Delete a file or directory
@@ -210,11 +211,14 @@ The header is two fixed entries and a variable length path:
 * Command: Single byte. Always ``0x40``.
 * 1 Byte reserved for padding.
 * Path length: 16-bit number encoding the encoded length of the path string.
+* Current time: 64-bit number encoding nanoseconds since January 1st, 1970. Used as the file modification time. Not all system will support the full resolution. Use the truncated time response value for caching.
 * Path: UTF-8 encoded string that is *not* null terminated. (We send the length instead.)
 
 The server will reply with:
 * Command: Single byte. Always ``0x41``.
 * Status: Single byte. ``0x01`` if the directory(s) were created or ``0x02`` if any parent of the path is an existing file.
+* 2 Bytes reserved for padding.
+* Truncated time: 64-bit number encoding nanoseconds since January 1st, 1970 as stored by the file system. The resolution may be less that the protocol. It is sent back for use in caching on the host side.
 
 ``0x50`` - List a directory
 +++++++++++++++++++++++++++
@@ -252,39 +256,37 @@ Moves a file or directory at a given path to a different path. Can be used to
 rename as well. The two paths are sent separately so that they are not limited
 by internal packet buffer sizes differently from other commands.
 
-**TODO** Maybe we do pack two paths in one packet. That will limit path length
-but prevent any secondary storage internally.
-
 The header is two fixed entries and a variable length path:
 
 * Command: Single byte. Always ``0x60``.
 * 1 Byte reserved for padding.
 * Old Path length: 16-bit number encoding the encoded length of the path string.
-* Old Path: UTF-8 encoded string that is *not* null terminated. (We send the length instead.)
-
-The server will reply with:
-* Command: Single byte. Always ``0x61``.
-* Status: Single byte. ``0x01`` on success or ``0x02`` if the old path is too long for internal buffers.
-
-* Command: Single byte. Always ``0x62``.
-* 1 Byte reserved for padding.
 * New Path length: 16-bit number encoding the encoded length of the path string.
+* Old Path: UTF-8 encoded string that is *not* null terminated. (We send the length instead.)
 * New Path: UTF-8 encoded string that is *not* null terminated. (We send the length instead.)
 
 The server will reply with:
-* Command: Single byte. Always ``0x63``.
+* Command: Single byte. Always ``0x61``.
 * Status: Single byte. ``0x01`` on success or ``0x02`` on error.
 
+**NOTE**: This is added in version 4.
+
 Versions
 =========
 
 Version 2
 ---------
+* Changes delete to delete contents of non-empty directories automatically.
 
+Version 3
+---------
 * Adds modification time.
   * Adds current time to file write command.
+  * Adds current time to make directory command.
   * Adds modification time to directory listing entries.
-* Changes delete to delete non-empty directories automatically.
+
+Version 4
+---------
 * Adds move command.
 
 Contributing

adafruit_ble_file_transfer.py

@@ -13,6 +13,7 @@
 """
 
 import struct
+import time
 import _bleio
 
 from adafruit_ble.attributes import Attribute
@@ -75,7 +76,7 @@ class FileTransferService(Service):
     # pylint: disable=too-few-public-methods
 
     uuid = StandardUUID(0xFEBB)
-    version = Uint32Characteristic(uuid=FileTransferUUID(0x0100))
+    version = Uint32Characteristic(uuid=FileTransferUUID(0x0100), initial_value=4)
     raw = _TransferCharacteristic()
     # _raw gets shadowed for each MIDIService instance by a PacketBuffer. PyLint doesn't know this
     # so it complains about missing members.
@@ -95,6 +96,8 @@ class FileTransferService(Service):
     MKDIR_STATUS = 0x41
     LISTDIR = 0x50
     LISTDIR_ENTRY = 0x51
+    MOVE = 0x60
+    MOVE_STATUS = 0x61
 
     # Responses
     # 0x00 is INVALID
@@ -204,24 +207,32 @@ def read(self, path, *, offset=0):
             self._write(encoded)
         return buf
 
-    def write(self, path, contents, *, offset=0):
+    def write(self, path, contents, *, offset=0, modification_time=None):
         """Writes the given contents to the given path starting at the given offset.
+        Returns the trunctated modification time.
 
         If the file is shorter than the offset, zeros will be added in the gap."""
         path = path.encode("utf-8")
         total_length = len(contents) + offset
+        if modification_time is None:
+            modification_time = int(time.time() * 1_000_000_000)
         encoded = (
             struct.pack(
-                "<BxHII", FileTransferService.WRITE, len(path), offset, total_length
+                "<BxHIIQ",
+                FileTransferService.WRITE,
+                len(path),
+                offset,
+                total_length,
+                modification_time,
             )
             + path
         )
         self._write(encoded)
-        b = bytearray(struct.calcsize("<BBxxII"))
+        b = bytearray(struct.calcsize("<BBxxIIQ"))
         written = 0
         while written < len(contents):
             self._readinto(b)
-            cmd, status, current_offset, free_space = struct.unpack("<BBxxII", b)
+            cmd, status, current_offset, free_space, _ = struct.unpack("<BBxxIIQ", b)
             if status != FileTransferService.OK:
                 print("write error", status)
                 raise RuntimeError()
@@ -254,23 +265,32 @@ def write(self, path, contents, *, offset=0):
 
         # Wait for confirmation that everything was written ok.
         self._readinto(b)
-        cmd, status, offset, free_space = struct.unpack("<BBxxII", b)
+        cmd, status, offset, free_space, truncated_time = struct.unpack("<BBxxIIQ", b)
         if cmd != FileTransferService.WRITE_PACING or offset != total_length:
             raise ProtocolError()
+        return truncated_time
 
-    def mkdir(self, path):
-        """Makes the directory and any missing parents"""
+    def mkdir(self, path, modification_time=None):
+        """Makes the directory and any missing parents. Returns the truncated time"""
         path = path.encode("utf-8")
-        encoded = struct.pack("<BxH", FileTransferService.MKDIR, len(path)) + path
+        if modification_time is None:
+            modification_time = int(time.time() * 1_000_000_000)
+        encoded = (
+            struct.pack(
+                "<BxHQ", FileTransferService.MKDIR, len(path), modification_time
+            )
+            + path
+        )
         self._write(encoded)
 
         b = bytearray(struct.calcsize("<BB"))
         self._readinto(b)
-        cmd, status = struct.unpack("<BB", b)
+        cmd, status, truncated_time = struct.unpack("<BBxxQ", b)
         if cmd != FileTransferService.MKDIR_STATUS:
             raise ProtocolError()
         if status != FileTransferService.OK:
             raise ValueError("Invalid path")
+        return truncated_time
 
     def listdir(self, path):
         """Returns a list of tuples, one tuple for each file or directory in the given path"""
@@ -282,22 +302,23 @@ def listdir(self, path):
         b = bytearray(self._service.raw.incoming_packet_length)
         i = 0
         total = 10  # starting value that will be replaced by the first response
-        header_size = struct.calcsize("<BBHIIII")
+        header_size = struct.calcsize("<BBHIIIIQ")
         path_length = 0
         encoded_path = b""
         while i < total:
             read = self._readinto(b)
             offset = 0
             file_size = 0
             flags = 0
+            modification_time = 0
             while offset < read:
                 if len(encoded_path) == path_length:
                     if path_length > 0:
                         path = str(
                             encoded_path,
                             "utf-8",
                         )
-                        paths.append((path, file_size, flags))
+                        paths.append((path, file_size, flags, modification_time))
                     (
                         cmd,
                         status,
@@ -306,7 +327,8 @@ def listdir(self, path):
                         total,
                         flags,
                         file_size,
-                    ) = struct.unpack_from("<BBHIIII", b, offset=offset)
+                        modification_time,
+                    ) = struct.unpack_from("<BBHIIIIQ", b, offset=offset)
                     offset += header_size
                     encoded_path = b""
                     if cmd != FileTransferService.LISTDIR_ENTRY:
@@ -322,7 +344,7 @@ def listdir(self, path):
         return paths
 
     def delete(self, path):
-        """Deletes the file or directory at the given path. Directories must be empty."""
+        """Deletes the file or directory at the given path."""
         path = path.encode("utf-8")
         encoded = struct.pack("<BxH", FileTransferService.DELETE, len(path)) + path
         self._write(encoded)
@@ -334,3 +356,22 @@ def delete(self, path):
             raise ProtocolError()
         if status != FileTransferService.OK:
             raise ValueError("Missing file")
+
+    def move(self, old_path, new_path):
+        """Moves the file or directory from old_path to new_path."""
+        old_path = old_path.encode("utf-8")
+        new_path = new_path.encode("utf-8")
+        encoded = (
+            struct.pack("<BxHH", FileTransferService.MOVE, len(old_path), len(new_path))
+            + old_path
+            + new_path
+        )
+        self._write(encoded)
+
+        b = bytearray(struct.calcsize("<BB"))
+        self._readinto(b)
+        cmd, status = struct.unpack("<BB", b)
+        if cmd != FileTransferService.MOVE_STATUS:
+            raise ProtocolError()
+        if status != FileTransferService.OK:
+            raise ValueError("Missing file")