Reverb Adders

Sequence Adder

class ftw.adders.reverb.SequenceAdder(client: <sphinx.ext.autodoc.importer._MockObject object at 0x7f0c8318d748>, sequence_length: int, period: int, delta_encoded: bool = False, chunk_length: Optional[int] = None, priority_fns: Optional[Mapping[str, Callable[[PriorityFnInput], float]]] = None, pad_end_of_episode: bool = True)

Bases: acme.adders.reverb.base.ReverbAdder

An adder which adds sequences of fixed length.

__init__(client: <sphinx.ext.autodoc.importer._MockObject object at 0x7f0c8318d748>, sequence_length: int, period: int, delta_encoded: bool = False, chunk_length: Optional[int] = None, priority_fns: Optional[Mapping[str, Callable[[PriorityFnInput], float]]] = None, pad_end_of_episode: bool = True)

Makes a SequenceAdder instance.

Args:

client: See docstring for BaseAdder. sequence_length: The fixed length of sequences we wish to add. period: The period with which we add sequences. If less than

sequence_length, overlapping sequences are added. If equal to sequence_length, sequences are exactly non-overlapping.
delta_encoded: If True (False by default) enables delta encoding, see
Client for more information.
chunk_length: Number of timesteps grouped together before delta encoding
and compression. See Client for more information.

priority_fns: See docstring for BaseAdder. pad_end_of_episode: If True (default) then upon end of episode the current

sequence will be padded (with observations, actions, etc… whose values are 0) until its length is sequence_length. If False then the last sequence in the episode may have length less than sequence_length.
reset()

Resets the adder’s buffer.

Multi-Sequence Adder

class ftw.adders.reverb.MultiSequenceAdder(client: <sphinx.ext.autodoc.importer._MockObject object at 0x7f0c5ad10080>, sequence_lengths: Mapping[str, int], periods: Mapping[str, int], delta_encoded: bool = False, chunk_length: Optional[int] = None, priority_fns: Optional[Mapping[str, Callable[[PriorityFnInput], float]]] = None, should_insert_fns: Optional[Mapping[str, Callable[[acme.adders.reverb.base.PriorityFnInput], bool]]] = None, pad_end_of_episode: bool = True)

Bases: acme.adders.reverb.base.ReverbAdder

An adder which adds multiple sequences of fixed lengths.

This class is based on acme.adders.reverb.SequenceAdder, with the following enhancements:
  • supports multiple sequences, for which the sequence lengths and periods may differ for each sequence.
  • supports conditional inserting of sequences, via the should_insert_fns argument of the constructor.

Furthermore, this class implements a similar bugfix (related to padding) as ftw.adders.reverb.SequenceAdder.

__init__(client: <sphinx.ext.autodoc.importer._MockObject object at 0x7f0c5ad10080>, sequence_lengths: Mapping[str, int], periods: Mapping[str, int], delta_encoded: bool = False, chunk_length: Optional[int] = None, priority_fns: Optional[Mapping[str, Callable[[PriorityFnInput], float]]] = None, should_insert_fns: Optional[Mapping[str, Callable[[acme.adders.reverb.base.PriorityFnInput], bool]]] = None, pad_end_of_episode: bool = True)

Makes a SequenceAdder instance.

Args:

client: See docstring for BaseAdder. sequence_lengths: Dictionary mapping table names

to the fixed length of sequences we wish to add for that table.
periods: Dictionary mapping table names
to the period with which we add sequences to that table. If less than sequence_length, overlapping sequences are added. If equal to sequence_length, sequences are exactly non-overlapping.
delta_encoded: If True (False by default) enables delta encoding, see
Client for more information.
chunk_length: Number of timesteps grouped together before delta encoding
and compression. See Client for more information.

priority_fns: See docstring for BaseAdder. should_insert_fns: Dictionary of functions for each sequence (table),

taking a PriorityFnInput NamedTuple as input and returning a bool indicating whether the sequence should be added to the corresponding table.
pad_end_of_episode: If True (default) then upon end of episode the current
sequence will be padded (with observations, actions, etc… whose values are 0) until its length is sequence_length. If False then the last sequence in the episode may have length less than sequence_length.
Raises:
ValueError: If invalid or conflicting arguments are passed to the constructor.
reset()

Resets the adder’s buffer.

Non-Overlapping RNN Sequence Adder

class ftw.adders.reverb.NonOverlappingRNNSequenceAdder(client: <sphinx.ext.autodoc.importer._MockObject object at 0x7f0c3a8f2c18>, sequence_length: int, delta_encoded: bool = False, chunk_length: Optional[int] = None, priority_fns: Optional[Mapping[str, Callable[[PriorityFnInput], float]]] = None, pad_end_of_episode: bool = True)

Bases: acme.adders.reverb.base.ReverbAdder

An adder which adds non-overlapping sequences of fixed length.

This adder is based on acme.adders.reverb.SequenceAdder, with the following modification: Enables adding sequences, where only the first recurrent core_state of the sequence is included. This saves RAM and can be used for any recurrent agent that is trained on unrolled sequences, if no overlapping of sequences is required.

Must be used in combination with ftw.datasets.make_reverb_rnn_sequence_fifo_sampler_dataset().

__init__(client: <sphinx.ext.autodoc.importer._MockObject object at 0x7f0c3a8f2c18>, sequence_length: int, delta_encoded: bool = False, chunk_length: Optional[int] = None, priority_fns: Optional[Mapping[str, Callable[[PriorityFnInput], float]]] = None, pad_end_of_episode: bool = True)

Makes a SequenceAdder instance.

Args:

client: See docstring for BaseAdder. sequence_length: The fixed length of sequences we wish to add. delta_encoded: If True (False by default) enables delta encoding, see

Client for more information.
chunk_length: Number of timesteps grouped together before delta encoding
and compression. See Client for more information.

priority_fns: See docstring for BaseAdder. pad_end_of_episode: If True (default) then upon end of episode the current

sequence will be padded (with observations, actions, etc… whose values are 0) until its length is sequence_length. If False then the last sequence in the episode may have length less than sequence_length.
reset()

Resets the adder’s buffer.