KPLRecordReader

class dragon.vm.dali.ops.KPLRecordReader(
  path,
  features,
  pipeline,
  shard_id=0,
  num_shards=1,
  random_shuffle=False,
  initial_fill=1024,
  **kwargs
)[source]

Read examples from the KPLRecord.

Examples:

class MyPipeline(dali.Pipeline):

    def __init__():
        super(MyPipeline, self).__init__()
        # Assume the we have the following data:
        # /data/root.data
        # /data/root.index
        # /data/root.meta
        self.reader = dali.ops.KPLRecordReader(
            path='/data'
            features=('image', 'label'),
            pipeline=self,
            # Shuffle locally in the next ``initial_fill`` examples
            # It turns to be weak with the decreasing of ``initial_fill``
            # and disabled if ``initial_fill`` is set to **1**
            random_shuffle=True,
            initial_fill=1024,
        )

    def iter_step(self):
        self.reader.feed_inputs()

    def define_graph(self):
        inputs = self.reader()

__init__

KPLRecordReader.__init__(
  path,
  features,
  pipeline,
  shard_id=0,
  num_shards=1,
  random_shuffle=False,
  initial_fill=1024,
  **kwargs
)[source]

Create a KPLRecordReader.

Parameters:
  • path (str) – The folder of record files.
  • features (Sequence[str], required) – The name of features to extract.
  • pipeline (nvidia.dali.Pipeline, required) – The pipeline to connect to.
  • shard_id (int, optional, default=0) – The index of partition to read.
  • num_shards (int, optional, default=1) – The total number of partitions over dataset.
  • random_shuffle (bool, optional, default=False) – Whether to shuffle the data.
  • initial_fill (int, optional, default=1024) – The length of sampling sequence for shuffle.

Methods

example_to_data

KPLRecordReader.example_to_data(example)[source]

Define the translation from example to array data.

Override this method to implement the translation.

feed_inputs

KPLRecordReader.feed_inputs()[source]

Feed the data to edge references.

Call this method in the Pipeline.iter_setup(...).

__call__

KPLRecordReader.__call__(
  *args,
  **kwargs
)[source]

Create the edge references for features.

Call this method in the Pipeline.define_graph(...).

Returns:
Dict[str, _EdgeReference] – The feature reference dict.