From 9f8c8da0b0b190b68a2177aba58889ace126fed8 Mon Sep 17 00:00:00 2001 From: Roger Taylor Date: Mon, 28 Jan 2019 09:07:38 +1300 Subject: [PATCH 1/3] Added draft segmented file protocol --- 0x-segmented-file.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 0x-segmented-file.md diff --git a/0x-segmented-file.md b/0x-segmented-file.md new file mode 100644 index 0000000..77b002c --- /dev/null +++ b/0x-segmented-file.md @@ -0,0 +1,22 @@ +This segmented file protocol is intended to allow a file to be reconstituted from multiple parts located on the blockchain. + +The idea of the segmented file protocol is to have a standard way to put files on the blockchain in OP_RETURN data, where the file is separated over multiple transactions or outputs. + +A high level outline of a segmented file is as follows: + +"sgfl" [filename] [segment-count] [data-reference] + +In detail these are: + +* "sgfl": A 4-byte prefix. In hex, it is 0x7367666c. +* [filename]: Pushdata containing a byte string. However, it is recommended that the filename be a utf8 character string. Furthermore, it is recommended that it be divided into name and extension like normal files: [name].[extension] . Operating systems are already used to dealing with filenames and parsing meaning from them, so we can reuse code from operating systems for this. +* [segment-count]: Pushdata containing a varint number, indicating the number of data-references that follow. +* [data-reference]: Pushdata containing a concatenated stream of, for each segment, a txid (32 bytes) and a varint tx output (variable length). + +An example of a segmented file is as follows: + +``` +OP_RETURN 4 0x7367666c 9 0x626c6164652e6a7067 1 0x03 103 22cd5757d5194daa014944128cbf86261a5f42e78c2f4e541b6b7774297e74b90222cd5757d5194daa014944128cbf86261a5f42e78c2f4e541b6b7774297e74b9fd122722cd5757d5194daa014944128cbf86261a5f42e78c2f4e541b6b7774297e74b9fd0002 +``` + +This is the file 'blade.jpg' comprised of three parts, all in the same transaction, but in segments located in outputs 2, 10002 and 512. \ No newline at end of file From 639b05d9610fe401521846ab02a345b3af534ff9 Mon Sep 17 00:00:00 2001 From: Roger Taylor Date: Mon, 28 Jan 2019 09:19:52 +1300 Subject: [PATCH 2/3] Remove not used segment count reference. --- 0x-segmented-file.md | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/0x-segmented-file.md b/0x-segmented-file.md index 77b002c..ff321f6 100644 --- a/0x-segmented-file.md +++ b/0x-segmented-file.md @@ -1,22 +1,35 @@ -This segmented file protocol is intended to allow a file to be reconstituted from multiple parts located on the blockchain. +# Segmented file protocol The idea of the segmented file protocol is to have a standard way to put files on the blockchain in OP_RETURN data, where the file is separated over multiple transactions or outputs. +## Protocol + A high level outline of a segmented file is as follows: -"sgfl" [filename] [segment-count] [data-reference] +"sgfl" [filename] [data-reference] In detail these are: * "sgfl": A 4-byte prefix. In hex, it is 0x7367666c. * [filename]: Pushdata containing a byte string. However, it is recommended that the filename be a utf8 character string. Furthermore, it is recommended that it be divided into name and extension like normal files: [name].[extension] . Operating systems are already used to dealing with filenames and parsing meaning from them, so we can reuse code from operating systems for this. -* [segment-count]: Pushdata containing a varint number, indicating the number of data-references that follow. * [data-reference]: Pushdata containing a concatenated stream of, for each segment, a txid (32 bytes) and a varint tx output (variable length). An example of a segmented file is as follows: ``` -OP_RETURN 4 0x7367666c 9 0x626c6164652e6a7067 1 0x03 103 22cd5757d5194daa014944128cbf86261a5f42e78c2f4e541b6b7774297e74b90222cd5757d5194daa014944128cbf86261a5f42e78c2f4e541b6b7774297e74b9fd122722cd5757d5194daa014944128cbf86261a5f42e78c2f4e541b6b7774297e74b9fd0002 +OP_RETURN 4 0x7367666c 9 0x626c6164652e6a7067 103 22cd5757d5194daa014944128cbf86261a5f42e78c2f4e541b6b7774297e74b90222cd5757d5194daa014944128cbf86261a5f42e78c2f4e541b6b7774297e74b9fd122722cd5757d5194daa014944128cbf86261a5f42e78c2f4e541b6b7774297e74b9fd0002 ``` -This is the file 'blade.jpg' comprised of three parts, all in the same transaction, but in segments located in outputs 2, 10002 and 512. \ No newline at end of file +This is the file 'blade.jpg' comprised of three parts, all in the same transaction, but in segments located in outputs 2, 10002 and 512. + +## Segment contents + +These are the ways segments are processed: + +* If the segment is recognised as being defined by this segmented file protocol, the reconstructed data is used as the segment data. +* If the segment is recognised as being defined by the file protocol, the embedded file data is used as the segment data. +* Otherwise the data within the first pushdata in the transaction output references as the segment, is used as the segment data. + +## On disk representation + +The complete OP_RETURN contents (excluding the leading OP_RETURN opcode) is stored in an on-disk file, perhaps as '.bitcoin'. It can be identified and processed using the same processing that identifies and processes the contents of an OP_RETURN. \ No newline at end of file From 12d8ec0b2ef210b238886384194d2fe3a3350e45 Mon Sep 17 00:00:00 2001 From: Roger Taylor Date: Mon, 28 Jan 2019 09:22:47 +1300 Subject: [PATCH 3/3] Fix use of angle brackets so the text is shown for '[filename].bitcoin'. --- 0x-segmented-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/0x-segmented-file.md b/0x-segmented-file.md index ff321f6..79e7eae 100644 --- a/0x-segmented-file.md +++ b/0x-segmented-file.md @@ -32,4 +32,4 @@ These are the ways segments are processed: ## On disk representation -The complete OP_RETURN contents (excluding the leading OP_RETURN opcode) is stored in an on-disk file, perhaps as '.bitcoin'. It can be identified and processed using the same processing that identifies and processes the contents of an OP_RETURN. \ No newline at end of file +The complete OP_RETURN contents (excluding the leading OP_RETURN opcode) is stored in an on-disk file, perhaps as '[filename].bitcoin'. It can be identified and processed using the same processing that identifies and processes the contents of an OP_RETURN. \ No newline at end of file