README
¶
STRUCTure EXtensions
structex provides annotation rules that extend Go structures for implementation of encoding and decoding of byte backed data frames.
structex aims to make it easier to represent industry standard data formats that can be directly embedded in code.
Example
Given the first three bytes T10 SCSI Primary Commands definition of a SCSI INQUIRY data
A structex annotated struct type declaration might be
package main
import (
"fmt"
"github.com/HewlettPackard/structex"
)
type SCSI_Standard_Inquiry struct {
PeripheralDeviceType uint8 `bitfield:"5"` // Byte 0
PeripheralQualifier uint8 `bitfield:"3"`
reserved0 uint8 `bitfield:"6,reserved"` // Byte 1
LU_Cong uint8 `bitfield:"1"`
RMB uint8 `bitfield:"1"`
Version uint8 // Byte 2
}
func main() {
var inquiry = new(SCSI_Standard_Inquiry)
// Perform IOCTL on device, returns byte reader to byte response
if err := structex.Decode(byteReader, inquiry); err != nil {
panic("a problem")
}
fmt.Printf("SCSI Peripheral Device Type: %d\n", inquiry.PeripheralDeviceType)
fmt.Printf("SCSI Peripheral Qualifier: %d\n", inquiry.PeripheralQualifier)
fmt.Printf("Inquiry Version: %#02x\n", inquiry.Version)
// etc
}
Annotation Format
Endianness
Little-endian (default) or big-endian tags support for integer and unsigned integer types
little:""
big:""
Bitfield
Bitfields define a structure field with an explicit size in bits. They are analogous to bit fields in the C specification.
bitfield:"[size][,reserved]"
`size`: Specifies the size, in bits, of the field.
`reserved`: Optional modifier that specifies the field contains reserved
bits and should be encoded as zeros.
Self-Described Layout
Many industry standards support dynamically sized return fields where the data layout is self described by other fields. To support such formats two annotations are provided.
sizeof:"[name][,relative]"
name Specifies that the field describes the size of `name` within the
structure. Should be used with slices.
During decoding, if field is non-zero, the field's value is
used to limit the number elements in the array or slice of
name `name`.
relative Optional modifier that specifies the value describing the size of
`name` is relative to the field offset within the structure.
This is often used in T10.org documentation
countof:"[name]"
name Specifies that the value describes the count of elements in
the `name` field.
During decoding, if field is non-zero, the field's value is
used to limit the number elements in the array or slice of
name `name`.
Truncation
Structex expects sufficient data for decoding the desired structure. When data structures are used to define a maximum size of the response buffer, you can use the truncate tag on an arry or slice to permit structex to truncate the returned data with what is provided by the source
buffer. If truncate is not specified and the provided buffer is of smaller size than the data structure, an io.EOF error is expected.
truncate:""
Alignment
Annotations can specified the byte-alignment requirement for structure fields. Analogous to the alignas specifier in C. Can only be applied to non-bitfield structure fields.
align:"[value]"
value An integer value specifying the byte alignment of the field.
Invalid non-zero alignments panic.
Full Tags
The tags documented above are abbreviated for ease of use; if desired, the full tag format is supported. This provides clarity that tags are part of the structex package, but it means typing more and using every sort of quote and backtic at your disposal.
Examples
`structex:"little"`
`structex:"big"`
`structex:"bitfield='3'"`
`structex:"bitfield='3,reserved'"`
`structex:"countOf='D'"`
`structex:"sizeOf='E'"`
`structex:"sizeOf='F,relative'"`
`structex:"align='8'"`
`structex:"truncate"`
Performance
structex places code readability ahead of any pack/unpack performance. Reflection is used to analyze the structure definitions, which itself carries heavy overhead. Parsing the tags also takes time.
For those looking for more performant code, consider code generation or a cache based solutions
Documentation
¶
Overview ¶
Copyright 2021 Hewlett Packard Enterprise Development LP
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Copyright 2021 Hewlett Packard Enterprise Development LP ¶
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Copyright 2021 Hewlett Packard Enterprise Development LP ¶
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Copyright 2021 Hewlett Packard Enterprise Development LP ¶
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Copyright 2021 Hewlett Packard Enterprise Development LP ¶
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Copyright 2021 Hewlett Packard Enterprise Development LP ¶
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Index ¶
- Variables
- func Decode(reader io.ByteReader, s interface{}) error
- func DecodeByteBuffer(b *bytes.Buffer, s interface{}) error
- func Encode(writer io.ByteWriter, s interface{}) error
- func EncodeByteBuffer(s interface{}) ([]byte, error)
- func Size(s interface{}) (uint64, error)
- type Buffer
- type TaggingError
Constants ¶
This section is empty.
Variables ¶
var (
CannotDeductSliceLengthError = errors.New("Cannot duduct slice length")
)
Functions ¶
func Decode ¶
func Decode(reader io.ByteReader, s interface{}) error
Decode reads data from a ByteReader into provided annotated structure.
Deserialization occurs according to the annotations in the structure which take several options:
Bitfields:
Bitfields define a structure field with an explicit size in bits. They are
analogous to bit fields in the C specification. Un
`bitfield:[size][,reserved]`
size Specifies the size, in bits, of the field.
reserved Optional modifier that specifies the field contains reserved
bits and should be encoded as zeros.
Dynamic Layouts:
Many industry standards support dynamically sized return fields where the data layout is self described by other fields. To support such formats two annotations are provided. `sizeOf:"[Field][,relative]"` Field Specifies that the field describes the size of Field within the structure. During decoding, if field is non-zero, the field's value is used to limit the number elements in the array or slice of name Field. relative Optional modifier that specifies the field value describing the size of Field is relative to the field offset within the structure. This is often used in T10.org documentation `countOf:"[Field]"` Field Specifies that the field describes the count of elements in Field. During decoding, if field is non-zero, the field's value is used to limit the number elements in the array or slice of name Field.
Alignment:
Annotations can specified the byte-alignment requirement for structure fields. Analogous to the alignas specifier in C. Can only be applied to non-bitfield structure fields. `align:"[value]"` value An integer value specifying the byte alignment of the field. Invalid non-zero alignments panic.
func DecodeByteBuffer ¶
DecodeByteBuffer takes a raw byte buffer and unpacks the buffer into the provided structure. Unused bytes do not cause an error.
func Encode ¶
func Encode(writer io.ByteWriter, s interface{}) error
Encode serializes the data structure defined by 's' into the available io.ByteWriter stream. Annotation rules are as defined in the Decode function.
func EncodeByteBuffer ¶ added in v1.0.1
EncodeByteBuffer serializes the provided data structure 's' into a new byte buffer. Bytes are packed according to the annotation rules defined for 's'.
Types ¶
type Buffer ¶
type Buffer struct {
// contains filtered or unexported fields
}
Buffer forms the basis of the storage area used for Pack and Unpack operations.
func NewBuffer ¶
func NewBuffer(s interface{}) *Buffer
NewBuffer returns a new Buffer for making Pack and Unpack operations ahead of the file Write and after file Read operations, respectfully.
The general pattern is to declare an annotated structure and fill in the desired fields. For example
type Test struct {
Param1 uint32 `bitfield:"5"`
Param2 uint32 `bitfield:"3"`
}
would declare variable test with parameters
var test Test test.Param1 = 3 test.Param2 = 1
A buffer is allocated of appropriate size using the NewBuffer function
b := structex.NewBuffer(test)
for which the structure is packed into
structex.Pack(b, test)
The Buffer can then be used for a File Write operations
f, err := os.Open("path/to/file")
nbytes, err := f.Write(b.Bytes())
or used for File Read operations
nbytes, err := f.Read(b.Bytes())
for which the Buffer can feed into Unpack to decode the available bytes
var unpacked = new (Test) err := structex.Unpack(b, unpacked)
func (*Buffer) DebugDump ¶
func (buf *Buffer) DebugDump()
DebugDump will print the Buffer in raw byte format, 16 bytes per line. Useful for debugging structure packing and unpacking.
type TaggingError ¶
type TaggingError struct {
// contains filtered or unexported fields
}
A TaggingError occurs when the pack/unpack routines have detected the structure field annotation does not match what is expected in the data stream.
func (*TaggingError) Error ¶
func (e *TaggingError) Error() string
Source Files