dldataset

package module

v0.0.0-...-c28e4f8 Latest Latest Go to latest Published: Nov 19, 2018 License: NCSA Imports: 11 Imported by: 3

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/rai-project/dldataset

Links

Open Source Insights

README ¶

DLDataset

Download the ImageNet dataset

The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) dataset has 1000 categories and 1.2 million images. The images do not need to be preprocessed or packaged in any database, but the validation images need to be moved into appropriate subfolders.

Download the images from http://image-net.org/download-images
Extract the training data:

mkdir train && mv ILSVRC2012_img_train.tar train/ && cd train
tar -xvf ILSVRC2012_img_train.tar && rm -f ILSVRC2012_img_train.tar
find . -name "*.tar" | while read NAME ; do mkdir -p "${NAME%.tar}"; tar -xvf "${NAME}" -C "${NAME%.tar}"; rm -f "${NAME}"; done
cd ..

Extract the validation data and move images to subfolders:

mkdir val && mv ILSVRC2012_img_val.tar val/ && cd val && tar -xvf ILSVRC2012_img_val.tar
wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash

Todo

ImageNet Validation Dataset
http://mscoco.org/dataset/

Documentation ¶

Index ¶

Variables
func Datasets() []string
func Register(d Dataset)
type Dataset
- func Get(category, name string) (Dataset, error)
type LabeledData

Constants ¶

This section is empty.

Variables ¶

View Source

var (
	// Config holds the data read by rai-project/config
	Config = &dldatasetConfig{
		done: make(chan struct{}),
	}
)

Config ...

Functions ¶

func Datasets ¶

func Datasets() []string

Datasets ...

func Register ¶

func Register(d Dataset)

Register ...

Types ¶

type Dataset ¶

type Dataset interface {
	New(ctx context.Context) (Dataset, error)
	Category() string
	Name() string
	CanonicalName() string
	Download(ctx context.Context) error
	List(ctx context.Context) ([]string, error)
	Load(ctx context.Context) error
	Get(ctx context.Context, name string) (LabeledData, error)
	Next(ctx context.Context) (LabeledData, error)
	io.Closer
}

Dataset ...

func Get ¶

func Get(category, name string) (Dataset, error)

Get ...

type LabeledData ¶

type LabeledData interface {
	Label() string
	Data() (interface{}, error)
}

LabeledData ...

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
reader
vision

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL