Documentation¶
NumPy¶
- class kymatio.numpy.HarmonicScattering3D(J, shape, L=3, sigma_0=1, max_order=2, rotation_covariant=True, method='integral', points=None, integral_powers=(0.5, 1.0, 2.0), backend='numpy')¶
Bases:
ScatteringNumPy,ScatteringBase3DThe 3D solid harmonic scattering transform
This class implements solid harmonic scattering on a 3D input image. For details see https://arxiv.org/abs/1805.00571.
Example
# Set the parameters of the scattering transform. J = 3 M, N, O = 32, 32, 32 # Generate a sample signal. x = np.random.randn(M, N, O) # Define a HarmonicScattering3D object. S = HarmonicScattering3D(J, (M, N, O)) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S(x)
- Parameters:
J (int) – Number of scales.
shape (tuple of ints) – Shape (M, N, O) of the input signal
L (int, optional) – Number of l values. Defaults to 3.
sigma_0 (float, optional) – Bandwidth of mother wavelet. Defaults to 1.
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
rotation_covariant (bool, optional) –
If set to True the first-order moduli take the form:
\(\sqrt{\sum_m (x \star \psi_{j,l,m})^2)}\)
if set to False the first-order moduli take the form:
\(x \star \psi_{j,l,m}\)
The second order moduli change analogously. Defaults to True.
method (string, optional) – Specifies the method for obtaining scattering coefficients. Currently, only ‘integral’ is available. Defaults to ‘integral’.
integral_powers (array-like) – List of exponents to the power of which moduli are raised before integration.
- build()¶
Defines elementary routines.
This function should always call and create the filters via self.create_filters() defined below. For instance, via: self.filters = self.create_filters()
- scattering(input_array)¶
Apply the scattering transform
- Parameters:
input_array (np.ndarray) – Input of size (batch_size, M, N, O).
- Returns:
output – If max_order is 1 it returns an np.ndarray with the first-order scattering coefficients. If max_order is 2 it returns an np.ndarray with the first- and second- order scattering coefficients, concatenated along the feature axis.
- Return type:
np.ndarray
- class kymatio.numpy.Scattering1D(J, shape, Q=1, T=None, max_order=2, average=None, oversampling=0, out_type='array', backend='numpy')¶
Bases:
ScatteringNumPy,ScatteringBase1DThe 1D scattering transform
The scattering transform computes a cascade of wavelet transforms alternated with a complex modulus non-linearity. The scattering transform of a 1D signal \(x(t)\) may be written as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x(t) = x \star \phi_J(t)\),
\(S_J^{(1)} x(t, \lambda) = |x \star \psi_\lambda^{(1)}| \star \phi_J\), and
\(S_J^{(2)} x(t, \lambda, \mu) = |\,| x \star \psi_\lambda^{(1)}| \star \psi_\mu^{(2)} | \star \phi_J\).
In the above formulas, \(\star\) denotes convolution in time. The filters \(\psi_\lambda^{(1)}(t)\) and \(\psi_\mu^{(2)}(t)\) are analytic wavelets with center frequencies \(\lambda\) and \(\mu\), while \(\phi_J(t)\) is a real lowpass filter centered at the zero frequency.
The Scattering1D class implements the 1D scattering transform for a given set of filters whose parameters are specified at initialization. While the wavelets are fixed, other parameters may be changed after the object is created, such as whether to compute all of \(S_J^{(0)} x\), \(S_J^{(1)} x\), and \(S_J^{(2)} x\) or just \(S_J^{(0)} x\) and \(S_J^{(1)} x\).
Given an input np.ndarray x of shape (B, N), where B is the number of signals to transform (the batch size) and N is the length of the signal, we compute its scattering transform by passing it to the scattering method (or calling the alias __call__). Note that B can be one, in which case it may be omitted, giving an input of shape (N,).
# Set the parameters of the scattering transform. J = 6 N = 2 ** 13 Q = 8 # Generate a sample signal. x = np.random.randn(N) # Define a Scattering1D object. S = Scattering1D(J, N, Q) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S(x)
Above, the length of the signal is \(N = 2^{13} = 8192\), while the maximum scale of the scattering transform is set to \(2^J = 2^6 = 64\). The time-frequency resolution of the first-order wavelets \(\psi_\lambda^{(1)}(t)\) is set to Q = 8 wavelets per octave. The second-order wavelets \(\psi_\mu^{(2)}(t)\) always have one wavelet per octave.
- Jint
The maximum log-scale of the scattering transform. In other words, the maximum scale is given by \(2^J\).
- shapeint
The length of the input signals.
- Qint or tuple
By default, Q (int) is the number of wavelets per octave for the first order and that for the second order has one wavelet per octave. This default value can be modified by passing Q as a tuple with two values, i.e. Q = (Q1, Q2), where Q1 and Q2 are the number of wavelets per octave for the first and second order, respectively.
- Tint
temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
- max_orderint, optional
The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
- averageboolean, optional
Determines whether the output is averaged in time or not. The averaged output corresponds to the standard scattering transform, while the un-averaged output skips the last convolution by \(\phi_J(t)\). This parameter may be modified after object creation. Defaults to True. Deprecated in v0.3 in favour of T and will be removed in v0.4. Replace average=False by T=0 and set T>1 or leave T=None for average=True (default).
- oversamplinginteger >= 0, optional
Controls the oversampling factor relative to the default as a power of two. Since the convolving by wavelets (or lowpass filters) and taking the modulus reduces the high-frequency content of the signal, we can subsample to save space and improve performance. However, this may reduce precision in the calculation. If this is not desirable, oversampling can be set to a large value to prevent too much subsampling. This parameter may be modified after object creation. Defaults to 0.
- vectorizeboolean, optional
Determines wheter to return a vectorized scattering transform (that is, a large array containing the output) or a dictionary (where each entry corresponds to a separate scattering coefficient). This parameter may be modified after object creation. Deprecated in favor of out_type (see below). Defaults to True.
- out_typestr, optional
The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering coefficient with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.
- Jint
The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2 ** J.
- shapeint
The length of the input signals.
- Qint
The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave).
- Tint
temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
- pad_leftint
The amount of padding to the left of the signal.
- pad_rightint
The amount of padding to the right of the signal.
- phi_fdictionary
A dictionary containing the lowpass filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- psi1_fdictionary
A dictionary containing all the first-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- psi2_fdictionary
A dictionary containing all the second-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- max_orderint
The maximum scattering order of the transform.
- averageboolean
Controls whether the output should be averaged (the standard scattering transform) or not (resulting in wavelet modulus coefficients). Note that to obtain unaveraged output, the vectorize flag must be set to False or out_type must be set to ‘list’. Deprecated in favor of T. For more details, see the documentation for scattering.
- oversamplingint
The number of powers of two to oversample the output compared to the default subsampling rate determined from the filters.
- vectorizeboolean
Controls whether the output should be vectorized into a single Tensor or collected into a dictionary. Deprecated in favor of out_type. For more details, see the documentation for scattering.
- out_typestr
Specifices the output format of the transform, which is currently one of ‘array’ or ‘list’. If ‘array’, the output is a large array containing the scattering coefficients. If ‘list’, the output is a list of dictionaries, each containing a scattering coefficient along with meta information. For more information, see the documentation for scattering.
- scattering(x)¶
Apply the scattering transform
Given an input np.ndarray of size (B, N), where B is the batch size (it can be potentially an integer or a shape) and N is the length of the individual signals, this function computes its scattering transform. If the vectorize flag is set to True (or if it is not available in this frontend), the output is in the form of a np.ndarray or size (B, C, N1), where N1 is the signal length after subsampling to the scale \(2^J\) (with the appropriate oversampling factor to reduce aliasing), and C is the number of scattering coefficients. If vectorize is set False, however, the output is a dictionary containing C keys, each a tuple whose length corresponds to the scattering order and whose elements are the sequence of filter indices used.
Note that the vectorize flag has been deprecated in favor of the out_type parameter. If this is set to ‘array’ (the default), the vectorize flag is still respected, but if not, out_type takes precedence. The two current output types are ‘array’ and ‘list’. The former gives the type of output described above. If set to ‘list’, however, the output is a list of dictionaries, each dictionary corresponding to a scattering coefficient and its associated meta information. The coefficient is stored under the ‘coef’ key, while other keys contain additional information, such as ‘j’ (the scale of the filter used) and ‘n’ (the filter index).
Furthermore, if the average flag is set to False, these outputs are not averaged, but are simply the wavelet modulus coefficients of the filters.
- Parameters:
x (np.ndarray) – An input np.ndarray of size (B, N).
- Returns:
S – If out_type is ‘array’ and the vectorize flag is True, the output is an np.ndarray containing the scattering coefficients, while if vectorize is False, it is a dictionary indexed by tuples of filter indices. If out_type is ‘list’, the output is a list of dictionaries as described above.
- Return type:
tensor or dictionary
- class kymatio.numpy.Scattering2D(J, shape, L=8, max_order=2, pre_pad=False, backend='numpy', out_type='array')¶
Bases:
ScatteringNumPy,ScatteringBase2DThe 2D scattering transform
The scattering transform computes two wavelet transform followed by modulus non-linearity. It can be summarized as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x = x \star \phi_J\),
\(S_J^{(1)} x = [|x \star \psi^{(1)}_\lambda| \star \phi_J]_\lambda\), and
\(S_J^{(2)} x = [||x \star \psi^{(1)}_\lambda| \star \psi^{(2)}_\mu| \star \phi_J]_{\lambda, \mu}\).
where \(\star\) denotes the convolution (in space), \(\phi_J\) is a lowpass filter, \(\psi^{(1)}_\lambda\) is a family of bandpass filters and \(\psi^{(2)}_\mu\) is another family of bandpass filters. Only Morlet filters are used in this implementation. Convolutions are efficiently performed in the Fourier domain.
Example
# Set the parameters of the scattering transform. J = 3 M, N = 32, 32 # Generate a sample signal. x = np.random.randn(M, N) # Define a Scattering2D object. S = Scattering2D(J, (M, N)) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S(x)
- Parameters:
J (int) – Log-2 of the scattering scale.
shape (tuple of ints) – Spatial support (M, N) of the input
L (int, optional) – Number of angles used for the wavelet transform. Defaults to 8.
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
pre_pad (boolean, optional) – Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally. Defaults to False.
backend (object, optional) – Controls the backend which is combined with the frontend.
out_type (str, optional) – The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering path with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.
- J¶
Log-2 of the scattering scale.
- Type:
int
- shape¶
Spatial support (M, N) of the input
- Type:
tuple of ints
- L¶
Number of angles used for the wavelet transform.
- Type:
int, optional
- max_order¶
The maximum order of scattering coefficients to compute. Must be either 1 or 2.
- Type:
int, optional
- pre_pad¶
Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally.
- Type:
boolean
- Psi¶
Contains the wavelets filters at all resolutions. See filter_bank.filter_bank for an exact description.
- Type:
dictionary
- Phi¶
Contains the low-pass filters at all resolutions. See filter_bank.filter_bank for an exact description.
- Type:
dictionary
- M_padded, N_padded
Spatial support of the padded input.
- Type:
int
- out_type¶
The format of the scattering output. See documentation for out_type parameter above and the documentation for scattering.
- Type:
str
Notes
The design of the filters is optimized for the value L = 8.
The pre_pad flag is particularly useful when cropping bigger images because this does not introduce border effects inherent to padding.
- scattering(input)¶
Apply the scattering transform
- Parameters:
input (np.ndarray) – An input np.ndarray of size (B, M, N).
- Raises:
RuntimeError – In the event that the input does not have at least two dimensions, or the tensor is not contiguous, or the tensor is not of the correct spatial size, padded or not.
TypeError – In the event that the input is not of type np.ndarray.
- Returns:
S – Scattering transform of the input. If out_type is set to ‘array’ (or if it is not availabel for this frontend), this is an np.ndarray of shape (B, C, M1, N1) where M1 = M // 2 ** J and N1 = N // 2 ** J. The C is the number of scattering channels calculated. If out_type is ‘list’, the output is a list of dictionaries, with each dictionary corresponding to a scattering coefficient and its meta information. The actual coefficient is contained in the ‘coef’ key, while other keys hold additional information, such as ‘j’ (the scale of the filter used), and ‘theta’ (the angle index of the filter used).
- Return type:
np.ndarray
Scikit-learn¶
- class kymatio.sklearn.HarmonicScattering3D(J, shape, L=3, sigma_0=1, max_order=2, rotation_covariant=True, method='integral', points=None, integral_powers=(0.5, 1.0, 2.0), backend='numpy')¶
Bases:
ScatteringTransformerMixin,HarmonicScatteringNumPy3DThe 3D solid harmonic scattering transform
This class implements solid harmonic scattering on a 3D input image. For details see https://arxiv.org/abs/1805.00571.
This class inherits from BaseEstimator and TransformerMixin in sklearn.base. As a result, it supports calculating the scattering transform by calling the predict and transform methods. By extension, it can be included as part of a scikit-learn Pipeline.
Example
# Set the parameters of the scattering transform. J = 3 M, N, O = 32, 32, 32 # Generate a sample signal. x = np.random.randn(np.prod(M, N, O)) # Define a HarmonicScattering3D object. S = HarmonicScattering3D(J, (M, N, O)) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S.predict(x)
- Parameters:
J (int) – Number of scales.
shape (tuple of ints) – Shape (M, N, O) of the input signal
L (int, optional) – Number of l values. Defaults to 3.
sigma_0 (float, optional) – Bandwidth of mother wavelet. Defaults to 1.
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
rotation_covariant (bool, optional) –
If set to True the first-order moduli take the form:
\(\sqrt{\sum_m (x \star \psi_{j,l,m})^2)}\)
if set to False the first-order moduli take the form:
\(x \star \psi_{j,l,m}\)
The second order moduli change analogously. Defaults to True.
method (string, optional) – Specifies the method for obtaining scattering coefficients. Currently, only ‘integral’ is available. Defaults to ‘integral’.
integral_powers (array-like) – List of exponents to the power of which moduli are raised before integration.
- class kymatio.sklearn.Scattering1D(J, shape, Q=1, T=None, max_order=2, average=None, oversampling=0, out_type='array', backend='numpy')¶
Bases:
ScatteringTransformerMixin,ScatteringNumPy1DThe 1D scattering transform
The scattering transform computes a cascade of wavelet transforms alternated with a complex modulus non-linearity. The scattering transform of a 1D signal \(x(t)\) may be written as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x(t) = x \star \phi_J(t)\),
\(S_J^{(1)} x(t, \lambda) = |x \star \psi_\lambda^{(1)}| \star \phi_J\), and
\(S_J^{(2)} x(t, \lambda, \mu) = |\,| x \star \psi_\lambda^{(1)}| \star \psi_\mu^{(2)} | \star \phi_J\).
In the above formulas, \(\star\) denotes convolution in time. The filters \(\psi_\lambda^{(1)}(t)\) and \(\psi_\mu^{(2)}(t)\) are analytic wavelets with center frequencies \(\lambda\) and \(\mu\), while \(\phi_J(t)\) is a real lowpass filter centered at the zero frequency.
The Scattering1D class implements the 1D scattering transform for a given set of filters whose parameters are specified at initialization. While the wavelets are fixed, other parameters may be changed after the object is created, such as whether to compute all of \(S_J^{(0)} x\), \(S_J^{(1)} x\), and \(S_J^{(2)} x\) or just \(S_J^{(0)} x\) and \(S_J^{(1)} x\).
This class inherits from BaseEstimator and TransformerMixin in sklearn.base. As a result, it supports calculating the scattering transform by calling the predict and transform methods. By extension, it can be included as part of a scikit-learn Pipeline.
Given an input np.ndarray x of shape (B, N), where B is the number of signals to transform (the batch size) and N is the length of the signal, we compute its scattering transform by passing it to the scattering method (or calling the alias predict). Note that B can be one, in which case it may be omitted, giving an input of shape (N,).
Example
# Set the parameters of the scattering transform. J = 6 N = 2 ** 13 Q = 8 # Generate a sample signal. x = np.random.randn(np.prod(N)) # Define a Scattering1D object. S = Scattering1D(J, N, Q) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S.predict(x)
Above, the length of the signal is \(N = 2^{13} = 8192\), while the maximum scale of the scattering transform is set to \(2^J = 2^6 = 64\). The time-frequency resolution of the first-order wavelets \(\psi_\lambda^{(1)}(t)\) is set to Q = 8 wavelets per octave. The second-order wavelets \(\psi_\mu^{(2)}(t)\) always have one wavelet per octave.
- Parameters:
J (int) – The maximum log-scale of the scattering transform. In other words, the maximum scale is given by \(2^J\).
shape (int) – The length of the input signals.
Q (int or tuple) – By default, Q (int) is the number of wavelets per octave for the first order and that for the second order has one wavelet per octave. This default value can be modified by passing Q as a tuple with two values, i.e. Q = (Q1, Q2), where Q1 and Q2 are the number of wavelets per octave for the first and second order, respectively.
T (int) – temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
oversampling (integer >= 0, optional) – Controls the oversampling factor relative to the default as a power of two. Since the convolving by wavelets (or lowpass filters) and taking the modulus reduces the high-frequency content of the signal, we can subsample to save space and improve performance. However, this may reduce precision in the calculation. If this is not desirable, oversampling can be set to a large value to prevent too much subsampling. This parameter may be modified after object creation. Defaults to 0.
- J¶
The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2 ** J.
- Type:
int
- shape¶
The length of the input signals.
- Type:
int
- Q¶
The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave).
- Type:
int
- T¶
temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
- Type:
int
- pad_left¶
The amount of padding to the left of the signal.
- Type:
int
- pad_right¶
The amount of padding to the right of the signal.
- Type:
int
- phi_f¶
A dictionary containing the lowpass filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- Type:
dictionary
- psi1_f¶
A dictionary containing all the first-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- Type:
dictionary
- psi2_f¶
A dictionary containing all the second-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- Type:
dictionary
- max_order¶
The maximum scattering order of the transform.
- Type:
int
- oversampling¶
The number of powers of two to oversample the output compared to the default subsampling rate determined from the filters.
- Type:
int
- class kymatio.sklearn.Scattering2D(J, shape, L=8, max_order=2, pre_pad=False, backend='numpy', out_type='array')¶
Bases:
ScatteringTransformerMixin,ScatteringNumPy2DThe 2D scattering transform
The scattering transform computes two wavelet transform followed by modulus non-linearity. It can be summarized as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x = x \star \phi_J\),
\(S_J^{(1)} x = [|x \star \psi^{(1)}_\lambda| \star \phi_J]_\lambda\), and
\(S_J^{(2)} x = [||x \star \psi^{(1)}_\lambda| \star \psi^{(2)}_\mu| \star \phi_J]_{\lambda, \mu}\).
where \(\star\) denotes the convolution (in space), \(\phi_J\) is a lowpass filter, \(\psi^{(1)}_\lambda\) is a family of bandpass filters and \(\psi^{(2)}_\mu\) is another family of bandpass filters. Only Morlet filters are used in this implementation. Convolutions are efficiently performed in the Fourier domain.
This class inherits from BaseEstimator and TransformerMixin in sklearn.base. As a result, it supports calculating the scattering transform by calling the predict and transform methods. By extension, it can be included as part of a scikit-learn Pipeline.
Example
# Set the parameters of the scattering transform. J = 3 M, N = 32, 32 # Generate a sample signal. x = np.random.randn(np.prod(M, N)) # Define a Scattering2D object. S = Scattering2D(J, (M, N)) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S.predict(x)
- Parameters:
J (int) – Log-2 of the scattering scale.
shape (tuple of ints) – Spatial support (M, N) of the input
L (int, optional) – Number of angles used for the wavelet transform. Defaults to 8.
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
pre_pad (boolean, optional) – Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally. Defaults to False.
backend (object, optional) – Controls the backend which is combined with the frontend.
- J¶
Log-2 of the scattering scale.
- Type:
int
- shape¶
Spatial support (M, N) of the input
- Type:
tuple of ints
- L¶
Number of angles used for the wavelet transform.
- Type:
int, optional
- max_order¶
The maximum order of scattering coefficients to compute. Must be either 1 or 2.
- Type:
int, optional
- pre_pad¶
Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally.
- Type:
boolean
- Psi¶
Contains the wavelets filters at all resolutions. See filter_bank.filter_bank for an exact description.
- Type:
dictionary
- Phi¶
Contains the low-pass filters at all resolutions. See filter_bank.filter_bank for an exact description.
- Type:
dictionary
- M_padded, N_padded
Spatial support of the padded input.
- Type:
int
Notes
The design of the filters is optimized for the value L = 8.
The pre_pad flag is particularly useful when cropping bigger images because this does not introduce border effects inherent to padding.
PyTorch¶
- class kymatio.torch.HarmonicScattering3D(J, shape, L=3, sigma_0=1, max_order=2, rotation_covariant=True, method='integral', points=None, integral_powers=(0.5, 1.0, 2.0), backend='torch')¶
Bases:
ScatteringTorch,ScatteringBase3DThe 3D solid harmonic scattering transform
This class implements solid harmonic scattering on a 3D input image. For details see https://arxiv.org/abs/1805.00571.
This class inherits from torch.nn.Module. As a result, it has all the same capabilities, including transferring the object to the GPU using the cuda or to methods. This object would then take GPU tensors as input and output the scattering coefficients of those tensors.
Example
# Set the parameters of the scattering transform. J = 3 M, N, O = 32, 32, 32 # Generate a sample signal. x = torch.randn(M, N, O) # Define a HarmonicScattering3D object. S = HarmonicScattering3D(J, (M, N, O)) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S.forward(x)
- Parameters:
J (int) – Number of scales.
shape (tuple of ints) – Shape (M, N, O) of the input signal
L (int, optional) – Number of l values. Defaults to 3.
sigma_0 (float, optional) – Bandwidth of mother wavelet. Defaults to 1.
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
rotation_covariant (bool, optional) –
If set to True the first-order moduli take the form:
\(\sqrt{\sum_m (x \star \psi_{j,l,m})^2)}\)
if set to False the first-order moduli take the form:
\(x \star \psi_{j,l,m}\)
The second order moduli change analogously. Defaults to True.
method (string, optional) – Specifies the method for obtaining scattering coefficients. Currently, only ‘integral’ is available. Defaults to ‘integral’.
integral_powers (array-like) – List of exponents to the power of which moduli are raised before integration.
- build()¶
Defines elementary routines.
This function should always call and create the filters via self.create_filters() defined below. For instance, via: self.filters = self.create_filters()
- register_filters()¶
This function should be called after filters are generated, saving those arrays as module buffers.
- scattering(input_array)¶
Apply the scattering transform
- Parameters:
input_array (torch.Tensor) – Input of size (batch_size, M, N, O).
- Returns:
output – If max_order is 1 it returns a torch.Tensor with the first-order scattering coefficients. If max_order is 2 it returns a torch.Tensor with the first- and second- order scattering coefficients, concatenated along the feature axis.
- Return type:
torch.Tensor
- class kymatio.torch.Scattering1D(J, shape, Q=1, T=None, max_order=2, average=None, oversampling=0, out_type='array', backend='torch')¶
Bases:
ScatteringTorch,ScatteringBase1DThe 1D scattering transform
The scattering transform computes a cascade of wavelet transforms alternated with a complex modulus non-linearity. The scattering transform of a 1D signal \(x(t)\) may be written as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x(t) = x \star \phi_J(t)\),
\(S_J^{(1)} x(t, \lambda) = |x \star \psi_\lambda^{(1)}| \star \phi_J\), and
\(S_J^{(2)} x(t, \lambda, \mu) = |\,| x \star \psi_\lambda^{(1)}| \star \psi_\mu^{(2)} | \star \phi_J\).
In the above formulas, \(\star\) denotes convolution in time. The filters \(\psi_\lambda^{(1)}(t)\) and \(\psi_\mu^{(2)}(t)\) are analytic wavelets with center frequencies \(\lambda\) and \(\mu\), while \(\phi_J(t)\) is a real lowpass filter centered at the zero frequency.
The Scattering1D class implements the 1D scattering transform for a given set of filters whose parameters are specified at initialization. While the wavelets are fixed, other parameters may be changed after the object is created, such as whether to compute all of \(S_J^{(0)} x\), \(S_J^{(1)} x\), and \(S_J^{(2)} x\) or just \(S_J^{(0)} x\) and \(S_J^{(1)} x\).
This class inherits from torch.nn.Module. As a result, it has all the same capabilities, including transferring the object to the GPU using the cuda or to methods. This object would then take GPU tensors as input and output the scattering coefficients of those tensors.
Given an input torch.Tensor x of shape (B, N), where B is the number of signals to transform (the batch size) and N is the length of the signal, we compute its scattering transform by passing it to the scattering method (or calling the alias forward). Note that B can be one, in which case it may be omitted, giving an input of shape (N,).
# Set the parameters of the scattering transform. J = 6 N = 2 ** 13 Q = 8 # Generate a sample signal. x = torch.randn(N) # Define a Scattering1D object. S = Scattering1D(J, N, Q) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S.forward(x)
Above, the length of the signal is \(N = 2^{13} = 8192\), while the maximum scale of the scattering transform is set to \(2^J = 2^6 = 64\). The time-frequency resolution of the first-order wavelets \(\psi_\lambda^{(1)}(t)\) is set to Q = 8 wavelets per octave. The second-order wavelets \(\psi_\mu^{(2)}(t)\) always have one wavelet per octave.
- Jint
The maximum log-scale of the scattering transform. In other words, the maximum scale is given by \(2^J\).
- shapeint
The length of the input signals.
- Qint or tuple
By default, Q (int) is the number of wavelets per octave for the first order and that for the second order has one wavelet per octave. This default value can be modified by passing Q as a tuple with two values, i.e. Q = (Q1, Q2), where Q1 and Q2 are the number of wavelets per octave for the first and second order, respectively.
- Tint
temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
- max_orderint, optional
The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
- averageboolean, optional
Determines whether the output is averaged in time or not. The averaged output corresponds to the standard scattering transform, while the un-averaged output skips the last convolution by \(\phi_J(t)\). This parameter may be modified after object creation. Defaults to True. Deprecated in v0.3 in favour of T and will be removed in v0.4. Replace average=False by T=0 and set T>1 or leave T=None for average=True (default).
- oversamplinginteger >= 0, optional
Controls the oversampling factor relative to the default as a power of two. Since the convolving by wavelets (or lowpass filters) and taking the modulus reduces the high-frequency content of the signal, we can subsample to save space and improve performance. However, this may reduce precision in the calculation. If this is not desirable, oversampling can be set to a large value to prevent too much subsampling. This parameter may be modified after object creation. Defaults to 0.
- vectorizeboolean, optional
Determines wheter to return a vectorized scattering transform (that is, a large array containing the output) or a dictionary (where each entry corresponds to a separate scattering coefficient). This parameter may be modified after object creation. Deprecated in favor of out_type (see below). Defaults to True.
- out_typestr, optional
The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering coefficient with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.
- Jint
The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2 ** J.
- shapeint
The length of the input signals.
- Qint
The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave).
- Tint
temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
- pad_leftint
The amount of padding to the left of the signal.
- pad_rightint
The amount of padding to the right of the signal.
- phi_fdictionary
A dictionary containing the lowpass filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- psi1_fdictionary
A dictionary containing all the first-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- psi2_fdictionary
A dictionary containing all the second-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- max_orderint
The maximum scattering order of the transform.
- averageboolean
Controls whether the output should be averaged (the standard scattering transform) or not (resulting in wavelet modulus coefficients). Note that to obtain unaveraged output, the vectorize flag must be set to False or out_type must be set to ‘list’. Deprecated in favor of T. For more details, see the documentation for scattering.
- oversamplingint
The number of powers of two to oversample the output compared to the default subsampling rate determined from the filters.
- vectorizeboolean
Controls whether the output should be vectorized into a single Tensor or collected into a dictionary. Deprecated in favor of out_type. For more details, see the documentation for scattering.
- out_typestr
Specifices the output format of the transform, which is currently one of ‘array’ or ‘list’. If ‘array’, the output is a large array containing the scattering coefficients. If ‘list’, the output is a list of dictionaries, each containing a scattering coefficient along with meta information. For more information, see the documentation for scattering.
- load_filters()¶
This function loads filters from the module’s buffer
- register_filters()¶
This function run the filterbank function that will create the filters as numpy array, and then, it saves those arrays as module’s buffers.
- scattering(x)¶
Apply the scattering transform
Given an input torch.Tensor of size (B, N), where B is the batch size (it can be potentially an integer or a shape) and N is the length of the individual signals, this function computes its scattering transform. If the vectorize flag is set to True (or if it is not available in this frontend), the output is in the form of a torch.Tensor or size (B, C, N1), where N1 is the signal length after subsampling to the scale \(2^J\) (with the appropriate oversampling factor to reduce aliasing), and C is the number of scattering coefficients. If vectorize is set False, however, the output is a dictionary containing C keys, each a tuple whose length corresponds to the scattering order and whose elements are the sequence of filter indices used.
Note that the vectorize flag has been deprecated in favor of the out_type parameter. If this is set to ‘array’ (the default), the vectorize flag is still respected, but if not, out_type takes precedence. The two current output types are ‘array’ and ‘list’. The former gives the type of output described above. If set to ‘list’, however, the output is a list of dictionaries, each dictionary corresponding to a scattering coefficient and its associated meta information. The coefficient is stored under the ‘coef’ key, while other keys contain additional information, such as ‘j’ (the scale of the filter used) and ‘n’ (the filter index).
Furthermore, if the average flag is set to False, these outputs are not averaged, but are simply the wavelet modulus coefficients of the filters.
- Parameters:
x (torch.Tensor) – An input torch.Tensor of size (B, N).
- Returns:
S – If out_type is ‘array’ and the vectorize flag is True, the output is a torch.Tensor containing the scattering coefficients, while if vectorize is False, it is a dictionary indexed by tuples of filter indices. If out_type is ‘list’, the output is a list of dictionaries as described above.
- Return type:
tensor or dictionary
- class kymatio.torch.Scattering2D(J, shape, L=8, max_order=2, pre_pad=False, backend='torch', out_type='array')¶
Bases:
ScatteringTorch,ScatteringBase2DThe 2D scattering transform
The scattering transform computes two wavelet transform followed by modulus non-linearity. It can be summarized as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x = x \star \phi_J\),
\(S_J^{(1)} x = [|x \star \psi^{(1)}_\lambda| \star \phi_J]_\lambda\), and
\(S_J^{(2)} x = [||x \star \psi^{(1)}_\lambda| \star \psi^{(2)}_\mu| \star \phi_J]_{\lambda, \mu}\).
where \(\star\) denotes the convolution (in space), \(\phi_J\) is a lowpass filter, \(\psi^{(1)}_\lambda\) is a family of bandpass filters and \(\psi^{(2)}_\mu\) is another family of bandpass filters. Only Morlet filters are used in this implementation. Convolutions are efficiently performed in the Fourier domain.
This class inherits from torch.nn.Module. As a result, it has all the same capabilities, including transferring the object to the GPU using the cuda or to methods. This object would then take GPU tensors as input and output the scattering coefficients of those tensors.
Example
# Set the parameters of the scattering transform. J = 3 M, N = 32, 32 # Generate a sample signal. x = torch.randn(M, N) # Define a Scattering2D object. S = Scattering2D(J, (M, N)) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S.forward(x)
- Parameters:
J (int) – Log-2 of the scattering scale.
shape (tuple of ints) – Spatial support (M, N) of the input
L (int, optional) – Number of angles used for the wavelet transform. Defaults to 8.
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
pre_pad (boolean, optional) – Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally. Defaults to False.
backend (object, optional) – Controls the backend which is combined with the frontend.
out_type (str, optional) – The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering path with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.
- J¶
Log-2 of the scattering scale.
- Type:
int
- shape¶
Spatial support (M, N) of the input
- Type:
tuple of ints
- L¶
Number of angles used for the wavelet transform.
- Type:
int, optional
- max_order¶
The maximum order of scattering coefficients to compute. Must be either 1 or 2.
- Type:
int, optional
- pre_pad¶
Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally.
- Type:
boolean
- Psi¶
Contains the wavelets filters at all resolutions. See filter_bank.filter_bank for an exact description.
- Type:
dictionary
- Phi¶
Contains the low-pass filters at all resolutions. See filter_bank.filter_bank for an exact description.
- Type:
dictionary
- M_padded, N_padded
Spatial support of the padded input.
- Type:
int
- out_type¶
The format of the scattering output. See documentation for out_type parameter above and the documentation for scattering.
- Type:
str
Notes
The design of the filters is optimized for the value L = 8.
The pre_pad flag is particularly useful when cropping bigger images because this does not introduce border effects inherent to padding.
- load_filters()¶
This function loads filters from the module’s buffers
- register_filters()¶
This function run the filterbank function that will create the filters as numpy array, and then, it saves those arrays as module’s buffers.
- scattering(input)¶
Apply the scattering transform
- Parameters:
input (torch.Tensor) – An input torch.Tensor of size (B, M, N).
- Raises:
RuntimeError – In the event that the input does not have at least two dimensions, or the tensor is not contiguous, or the tensor is not of the correct spatial size, padded or not.
TypeError – In the event that the input is not of type torch.Tensor.
- Returns:
S – Scattering transform of the input. If out_type is set to ‘array’ (or if it is not availabel for this frontend), this is a torch.Tensor of shape (B, C, M1, N1) where M1 = M // 2 ** J and N1 = N // 2 ** J. The C is the number of scattering channels calculated. If out_type is ‘list’, the output is a list of dictionaries, with each dictionary corresponding to a scattering coefficient and its meta information. The actual coefficient is contained in the ‘coef’ key, while other keys hold additional information, such as ‘j’ (the scale of the filter used), and ‘theta’ (the angle index of the filter used).
- Return type:
torch.Tensor
TensorFlow¶
- class kymatio.tensorflow.HarmonicScattering3D(*args: Any, **kwargs: Any)¶
Bases:
ScatteringTensorFlow,ScatteringBase3DThe 3D solid harmonic scattering transform
This class implements solid harmonic scattering on a 3D input image. For details see https://arxiv.org/abs/1805.00571.
This class inherits from tf.Module. As a result, it has all the same capabilities as a standard TensorFlow Module.
Example
# Set the parameters of the scattering transform. J = 3 M, N, O = 32, 32, 32 # Generate a sample signal. x = np.random.randn(M, N, O) # Define a HarmonicScattering3D object. S = HarmonicScattering3D(J, (M, N, O)) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S(x)
- Parameters:
J (int) – Number of scales.
shape (tuple of ints) – Shape (M, N, O) of the input signal
L (int, optional) – Number of l values. Defaults to 3.
sigma_0 (float, optional) – Bandwidth of mother wavelet. Defaults to 1.
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
rotation_covariant (bool, optional) –
If set to True the first-order moduli take the form:
\(\sqrt{\sum_m (x \star \psi_{j,l,m})^2)}\)
if set to False the first-order moduli take the form:
\(x \star \psi_{j,l,m}\)
The second order moduli change analogously. Defaults to True.
method (string, optional) – Specifies the method for obtaining scattering coefficients. Currently, only ‘integral’ is available. Defaults to ‘integral’.
integral_powers (array-like) – List of exponents to the power of which moduli are raised before integration.
- build()¶
Defines elementary routines.
This function should always call and create the filters via self.create_filters() defined below. For instance, via: self.filters = self.create_filters()
- scattering(x)¶
Apply the scattering transform
- Parameters:
input_array (tf.Tensor) – Input of size (batch_size, M, N, O).
- Returns:
output – If max_order is 1 it returns a tf.Tensor with the first-order scattering coefficients. If max_order is 2 it returns a tf.Tensor with the first- and second- order scattering coefficients, concatenated along the feature axis.
- Return type:
tf.Tensor
- class kymatio.tensorflow.Scattering1D(*args: Any, **kwargs: Any)¶
Bases:
ScatteringTensorFlow,ScatteringBase1DThe 1D scattering transform
The scattering transform computes a cascade of wavelet transforms alternated with a complex modulus non-linearity. The scattering transform of a 1D signal \(x(t)\) may be written as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x(t) = x \star \phi_J(t)\),
\(S_J^{(1)} x(t, \lambda) = |x \star \psi_\lambda^{(1)}| \star \phi_J\), and
\(S_J^{(2)} x(t, \lambda, \mu) = |\,| x \star \psi_\lambda^{(1)}| \star \psi_\mu^{(2)} | \star \phi_J\).
In the above formulas, \(\star\) denotes convolution in time. The filters \(\psi_\lambda^{(1)}(t)\) and \(\psi_\mu^{(2)}(t)\) are analytic wavelets with center frequencies \(\lambda\) and \(\mu\), while \(\phi_J(t)\) is a real lowpass filter centered at the zero frequency.
The Scattering1D class implements the 1D scattering transform for a given set of filters whose parameters are specified at initialization. While the wavelets are fixed, other parameters may be changed after the object is created, such as whether to compute all of \(S_J^{(0)} x\), \(S_J^{(1)} x\), and \(S_J^{(2)} x\) or just \(S_J^{(0)} x\) and \(S_J^{(1)} x\).
This class inherits from tf.Module. As a result, it has all the same capabilities as a standard TensorFlow Module.
Given an input tf.Tensor x of shape (B, N), where B is the number of signals to transform (the batch size) and N is the length of the signal, we compute its scattering transform by passing it to the scattering method (or calling the alias __call__). Note that B can be one, in which case it may be omitted, giving an input of shape (N,).
# Set the parameters of the scattering transform. J = 6 N = 2 ** 13 Q = 8 # Generate a sample signal. x = np.random.randn(N) # Define a Scattering1D object. S = Scattering1D(J, N, Q) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S(x)
Above, the length of the signal is \(N = 2^{13} = 8192\), while the maximum scale of the scattering transform is set to \(2^J = 2^6 = 64\). The time-frequency resolution of the first-order wavelets \(\psi_\lambda^{(1)}(t)\) is set to Q = 8 wavelets per octave. The second-order wavelets \(\psi_\mu^{(2)}(t)\) always have one wavelet per octave.
- Jint
The maximum log-scale of the scattering transform. In other words, the maximum scale is given by \(2^J\).
- shapeint
The length of the input signals.
- Qint or tuple
By default, Q (int) is the number of wavelets per octave for the first order and that for the second order has one wavelet per octave. This default value can be modified by passing Q as a tuple with two values, i.e. Q = (Q1, Q2), where Q1 and Q2 are the number of wavelets per octave for the first and second order, respectively.
- Tint
temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
- max_orderint, optional
The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
- averageboolean, optional
Determines whether the output is averaged in time or not. The averaged output corresponds to the standard scattering transform, while the un-averaged output skips the last convolution by \(\phi_J(t)\). This parameter may be modified after object creation. Defaults to True. Deprecated in v0.3 in favour of T and will be removed in v0.4. Replace average=False by T=0 and set T>1 or leave T=None for average=True (default).
- oversamplinginteger >= 0, optional
Controls the oversampling factor relative to the default as a power of two. Since the convolving by wavelets (or lowpass filters) and taking the modulus reduces the high-frequency content of the signal, we can subsample to save space and improve performance. However, this may reduce precision in the calculation. If this is not desirable, oversampling can be set to a large value to prevent too much subsampling. This parameter may be modified after object creation. Defaults to 0.
- vectorizeboolean, optional
Determines wheter to return a vectorized scattering transform (that is, a large array containing the output) or a dictionary (where each entry corresponds to a separate scattering coefficient). This parameter may be modified after object creation. Deprecated in favor of out_type (see below). Defaults to True.
- out_typestr, optional
The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering coefficient with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.
- Jint
The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2 ** J.
- shapeint
The length of the input signals.
- Qint
The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave).
- Tint
temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
- pad_leftint
The amount of padding to the left of the signal.
- pad_rightint
The amount of padding to the right of the signal.
- phi_fdictionary
A dictionary containing the lowpass filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- psi1_fdictionary
A dictionary containing all the first-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- psi2_fdictionary
A dictionary containing all the second-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- max_orderint
The maximum scattering order of the transform.
- averageboolean
Controls whether the output should be averaged (the standard scattering transform) or not (resulting in wavelet modulus coefficients). Note that to obtain unaveraged output, the vectorize flag must be set to False or out_type must be set to ‘list’. Deprecated in favor of T. For more details, see the documentation for scattering.
- oversamplingint
The number of powers of two to oversample the output compared to the default subsampling rate determined from the filters.
- vectorizeboolean
Controls whether the output should be vectorized into a single Tensor or collected into a dictionary. Deprecated in favor of out_type. For more details, see the documentation for scattering.
- out_typestr
Specifices the output format of the transform, which is currently one of ‘array’ or ‘list’. If ‘array’, the output is a large array containing the scattering coefficients. If ‘list’, the output is a list of dictionaries, each containing a scattering coefficient along with meta information. For more information, see the documentation for scattering.
- scattering(x)¶
Apply the scattering transform
Given an input tf.Tensor of size (B, N), where B is the batch size (it can be potentially an integer or a shape) and N is the length of the individual signals, this function computes its scattering transform. If the vectorize flag is set to True (or if it is not available in this frontend), the output is in the form of a tf.Tensor or size (B, C, N1), where N1 is the signal length after subsampling to the scale \(2^J\) (with the appropriate oversampling factor to reduce aliasing), and C is the number of scattering coefficients. If vectorize is set False, however, the output is a dictionary containing C keys, each a tuple whose length corresponds to the scattering order and whose elements are the sequence of filter indices used.
Note that the vectorize flag has been deprecated in favor of the out_type parameter. If this is set to ‘array’ (the default), the vectorize flag is still respected, but if not, out_type takes precedence. The two current output types are ‘array’ and ‘list’. The former gives the type of output described above. If set to ‘list’, however, the output is a list of dictionaries, each dictionary corresponding to a scattering coefficient and its associated meta information. The coefficient is stored under the ‘coef’ key, while other keys contain additional information, such as ‘j’ (the scale of the filter used) and ‘n’ (the filter index).
Furthermore, if the average flag is set to False, these outputs are not averaged, but are simply the wavelet modulus coefficients of the filters.
- Parameters:
x (tf.Tensor) – An input tf.Tensor of size (B, N).
- Returns:
S – If out_type is ‘array’ and the vectorize flag is True, the output is a tf.Tensor containing the scattering coefficients, while if vectorize is False, it is a dictionary indexed by tuples of filter indices. If out_type is ‘list’, the output is a list of dictionaries as described above.
- Return type:
tensor or dictionary
- class kymatio.tensorflow.Scattering2D(*args: Any, **kwargs: Any)¶
Bases:
ScatteringTensorFlow,ScatteringBase2DThe 2D scattering transform
The scattering transform computes two wavelet transform followed by modulus non-linearity. It can be summarized as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x = x \star \phi_J\),
\(S_J^{(1)} x = [|x \star \psi^{(1)}_\lambda| \star \phi_J]_\lambda\), and
\(S_J^{(2)} x = [||x \star \psi^{(1)}_\lambda| \star \psi^{(2)}_\mu| \star \phi_J]_{\lambda, \mu}\).
where \(\star\) denotes the convolution (in space), \(\phi_J\) is a lowpass filter, \(\psi^{(1)}_\lambda\) is a family of bandpass filters and \(\psi^{(2)}_\mu\) is another family of bandpass filters. Only Morlet filters are used in this implementation. Convolutions are efficiently performed in the Fourier domain.
This class inherits from tf.Module. As a result, it has all the same capabilities as a standard TensorFlow Module.
Example
# Set the parameters of the scattering transform. J = 3 M, N = 32, 32 # Generate a sample signal. x = np.random.randn(M, N) # Define a Scattering2D object. S = Scattering2D(J, (M, N)) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S(x)
- Parameters:
J (int) – Log-2 of the scattering scale.
shape (tuple of ints) – Spatial support (M, N) of the input
L (int, optional) – Number of angles used for the wavelet transform. Defaults to 8.
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
pre_pad (boolean, optional) – Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally. Defaults to False.
backend (object, optional) – Controls the backend which is combined with the frontend.
out_type (str, optional) – The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering path with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.
- J¶
Log-2 of the scattering scale.
- Type:
int
- shape¶
Spatial support (M, N) of the input
- Type:
tuple of ints
- L¶
Number of angles used for the wavelet transform.
- Type:
int, optional
- max_order¶
The maximum order of scattering coefficients to compute. Must be either 1 or 2.
- Type:
int, optional
- pre_pad¶
Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally.
- Type:
boolean
- Psi¶
Contains the wavelets filters at all resolutions. See filter_bank.filter_bank for an exact description.
- Type:
dictionary
- Phi¶
Contains the low-pass filters at all resolutions. See filter_bank.filter_bank for an exact description.
- Type:
dictionary
- M_padded, N_padded
Spatial support of the padded input.
- Type:
int
- out_type¶
The format of the scattering output. See documentation for out_type parameter above and the documentation for scattering.
- Type:
str
Notes
The design of the filters is optimized for the value L = 8.
The pre_pad flag is particularly useful when cropping bigger images because this does not introduce border effects inherent to padding.
- scattering(input)¶
Apply the scattering transform
- Parameters:
input (tf.Tensor) – An input tf.Tensor of size (B, M, N).
- Raises:
RuntimeError – In the event that the input does not have at least two dimensions, or the tensor is not contiguous, or the tensor is not of the correct spatial size, padded or not.
TypeError – In the event that the input is not of type tf.Tensor.
- Returns:
S – Scattering transform of the input. If out_type is set to ‘array’ (or if it is not availabel for this frontend), this is a tf.Tensor of shape (B, C, M1, N1) where M1 = M // 2 ** J and N1 = N // 2 ** J. The C is the number of scattering channels calculated. If out_type is ‘list’, the output is a list of dictionaries, with each dictionary corresponding to a scattering coefficient and its meta information. The actual coefficient is contained in the ‘coef’ key, while other keys hold additional information, such as ‘j’ (the scale of the filter used), and ‘theta’ (the angle index of the filter used).
- Return type:
tf.Tensor
Keras¶
- class kymatio.keras.Scattering1D(*args: Any, **kwargs: Any)¶
Bases:
ScatteringKeras,ScatteringBase1DThe 1D scattering transform
The scattering transform computes a cascade of wavelet transforms alternated with a complex modulus non-linearity. The scattering transform of a 1D signal \(x(t)\) may be written as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x(t) = x \star \phi_J(t)\),
\(S_J^{(1)} x(t, \lambda) = |x \star \psi_\lambda^{(1)}| \star \phi_J\), and
\(S_J^{(2)} x(t, \lambda, \mu) = |\,| x \star \psi_\lambda^{(1)}| \star \psi_\mu^{(2)} | \star \phi_J\).
In the above formulas, \(\star\) denotes convolution in time. The filters \(\psi_\lambda^{(1)}(t)\) and \(\psi_\mu^{(2)}(t)\) are analytic wavelets with center frequencies \(\lambda\) and \(\mu\), while \(\phi_J(t)\) is a real lowpass filter centered at the zero frequency.
The Scattering1D class implements the 1D scattering transform for a given set of filters whose parameters are specified at initialization. While the wavelets are fixed, other parameters may be changed after the object is created, such as whether to compute all of \(S_J^{(0)} x\), \(S_J^{(1)} x\), and \(S_J^{(2)} x\) or just \(S_J^{(0)} x\) and \(S_J^{(1)} x\).
This class inherits from tf.keras.layers.Layer. As a result, it has all the same capabilities as a standard Keras Layer.
Given an input tf.Tensor x of shape (B, N), where B is the number of signals to transform (the batch size) and N is the length of the signal, we compute its scattering transform by passing it to the scattering method (or calling the alias call). Note that B can be one, in which case it may be omitted, giving an input of shape (N,).
Example
# Set the parameters of the scattering transform. J = 6 N = 2 ** 13 Q = 8 # Generate a sample signal. x = np.random.randn(N) # Define a Scattering1D object. S = Scattering1D(J, Q) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S(x)
Above, the length of the signal is \(N = 2^{13} = 8192\), while the maximum scale of the scattering transform is set to \(2^J = 2^6 = 64\). The time-frequency resolution of the first-order wavelets \(\psi_\lambda^{(1)}(t)\) is set to Q = 8 wavelets per octave. The second-order wavelets \(\psi_\mu^{(2)}(t)\) always have one wavelet per octave.
- Parameters:
J (int) – The maximum log-scale of the scattering transform. In other words, the maximum scale is given by \(2^J\).
Q (int or tuple) – By default, Q (int) is the number of wavelets per octave for the first order and that for the second order has one wavelet per octave. This default value can be modified by passing Q as a tuple with two values, i.e. Q = (Q1, Q2), where Q1 and Q2 are the number of wavelets per octave for the first and second order, respectively.
T (int) – temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
oversampling (integer >= 0, optional) – Controls the oversampling factor relative to the default as a power of two. Since the convolving by wavelets (or lowpass filters) and taking the modulus reduces the high-frequency content of the signal, we can subsample to save space and improve performance. However, this may reduce precision in the calculation. If this is not desirable, oversampling can be set to a large value to prevent too much subsampling. This parameter may be modified after object creation. Defaults to 0.
- J¶
The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2 ** J.
- Type:
int
- Q¶
The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave).
- Type:
int
- T¶
temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
- Type:
int
- max_order¶
The maximum scattering order of the transform.
- Type:
int
- oversampling¶
The number of powers of two to oversample the output compared to the default subsampling rate determined from the filters.
- Type:
int
- build(input_shape)¶
Set up padding and filters
Certain internal data, such as the amount of padding and the wavelet filters to be used in the scattering transform, need to be computed from the parameters given during construction. This function is called automatically during object creation and no subsequent calls are therefore needed.
- class kymatio.keras.Scattering2D(*args: Any, **kwargs: Any)¶
Bases:
ScatteringKeras,ScatteringBase2DThe 2D scattering transform
The scattering transform computes two wavelet transform followed by modulus non-linearity. It can be summarized as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x = x \star \phi_J\),
\(S_J^{(1)} x = [|x \star \psi^{(1)}_\lambda| \star \phi_J]_\lambda\), and
\(S_J^{(2)} x = [||x \star \psi^{(1)}_\lambda| \star \psi^{(2)}_\mu| \star \phi_J]_{\lambda, \mu}\).
where \(\star\) denotes the convolution (in space), \(\phi_J\) is a lowpass filter, \(\psi^{(1)}_\lambda\) is a family of bandpass filters and \(\psi^{(2)}_\mu\) is another family of bandpass filters. Only Morlet filters are used in this implementation. Convolutions are efficiently performed in the Fourier domain.
This class inherits from tf.keras.layers.Layer. As a result, it has all the same capabilities as a standard Keras Layer.
Example
# Set the parameters of the scattering transform. J = 3 M, N = 32, 32 # Generate a sample signal. x = np.random.randn(M, N) # Define a Scattering2D object. S = Scattering2D(J) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S(x)
- Parameters:
J (int) – Log-2 of the scattering scale.
L (int, optional) – Number of angles used for the wavelet transform. Defaults to 8.
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
pre_pad (boolean, optional) – Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally. Defaults to False.
backend (object, optional) – Controls the backend which is combined with the frontend.
- J¶
Log-2 of the scattering scale.
- Type:
int
- L¶
Number of angles used for the wavelet transform.
- Type:
int, optional
- max_order¶
The maximum order of scattering coefficients to compute. Must be either 1 or 2.
- Type:
int, optional
- pre_pad¶
Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally.
- Type:
boolean
Notes
The design of the filters is optimized for the value L = 8.
The pre_pad flag is particularly useful when cropping bigger images because this does not introduce border effects inherent to padding.
- build(input_shape)¶
Defines elementary routines.
This function should always call and create the filters via self.create_filters() defined below. For instance, via: self.filters = self.create_filters()
Jax¶
- class kymatio.jax.HarmonicScattering3D(J, shape, L=3, sigma_0=1, max_order=2, rotation_covariant=True, method='integral', points=None, integral_powers=(0.5, 1.0, 2.0), backend='jax')¶
Bases:
ScatteringJax,HarmonicScatteringNumPy3DThe 3D solid harmonic scattering transform
This class implements solid harmonic scattering on a 3D input image. For details see https://arxiv.org/abs/1805.00571.
Example
# Set the parameters of the scattering transform. J = 3 M, N, O = 32, 32, 32 # Generate a sample signal. x = np.random.randn(M, N, O) # Define a HarmonicScattering3D object. S = HarmonicScattering3D(J, (M, N, O)) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S(x)
- Parameters:
J (int) – Number of scales.
shape (tuple of ints) – Shape (M, N, O) of the input signal
L (int, optional) – Number of l values. Defaults to 3.
sigma_0 (float, optional) – Bandwidth of mother wavelet. Defaults to 1.
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
rotation_covariant (bool, optional) –
If set to True the first-order moduli take the form:
\(\sqrt{\sum_m (x \star \psi_{j,l,m})^2)}\)
if set to False the first-order moduli take the form:
\(x \star \psi_{j,l,m}\)
The second order moduli change analogously. Defaults to True.
method (string, optional) – Specifies the method for obtaining scattering coefficients. Currently, only ‘integral’ is available. Defaults to ‘integral’.
integral_powers (array-like) – List of exponents to the power of which moduli are raised before integration.
- build()¶
Defines elementary routines.
This function should always call and create the filters via self.create_filters() defined below. For instance, via: self.filters = self.create_filters()
- class kymatio.jax.Scattering1D(J, shape, Q=1, T=None, max_order=2, average=None, oversampling=0, out_type='array', backend='jax')¶
Bases:
ScatteringJax,ScatteringNumPy1DThe 1D scattering transform
The scattering transform computes a cascade of wavelet transforms alternated with a complex modulus non-linearity. The scattering transform of a 1D signal \(x(t)\) may be written as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x(t) = x \star \phi_J(t)\),
\(S_J^{(1)} x(t, \lambda) = |x \star \psi_\lambda^{(1)}| \star \phi_J\), and
\(S_J^{(2)} x(t, \lambda, \mu) = |\,| x \star \psi_\lambda^{(1)}| \star \psi_\mu^{(2)} | \star \phi_J\).
In the above formulas, \(\star\) denotes convolution in time. The filters \(\psi_\lambda^{(1)}(t)\) and \(\psi_\mu^{(2)}(t)\) are analytic wavelets with center frequencies \(\lambda\) and \(\mu\), while \(\phi_J(t)\) is a real lowpass filter centered at the zero frequency.
The Scattering1D class implements the 1D scattering transform for a given set of filters whose parameters are specified at initialization. While the wavelets are fixed, other parameters may be changed after the object is created, such as whether to compute all of \(S_J^{(0)} x\), \(S_J^{(1)} x\), and \(S_J^{(2)} x\) or just \(S_J^{(0)} x\) and \(S_J^{(1)} x\).
Given an input np.ndarray x of shape (B, N), where B is the number of signals to transform (the batch size) and N is the length of the signal, we compute its scattering transform by passing it to the scattering method (or calling the alias __call__). Note that B can be one, in which case it may be omitted, giving an input of shape (N,).
# Set the parameters of the scattering transform. J = 6 N = 2 ** 13 Q = 8 # Generate a sample signal. x = np.random.randn(N) # Define a Scattering1D object. S = Scattering1D(J, N, Q) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S(x)
Above, the length of the signal is \(N = 2^{13} = 8192\), while the maximum scale of the scattering transform is set to \(2^J = 2^6 = 64\). The time-frequency resolution of the first-order wavelets \(\psi_\lambda^{(1)}(t)\) is set to Q = 8 wavelets per octave. The second-order wavelets \(\psi_\mu^{(2)}(t)\) always have one wavelet per octave.
- Jint
The maximum log-scale of the scattering transform. In other words, the maximum scale is given by \(2^J\).
- shapeint
The length of the input signals.
- Qint or tuple
By default, Q (int) is the number of wavelets per octave for the first order and that for the second order has one wavelet per octave. This default value can be modified by passing Q as a tuple with two values, i.e. Q = (Q1, Q2), where Q1 and Q2 are the number of wavelets per octave for the first and second order, respectively.
- Tint
temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
- max_orderint, optional
The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
- averageboolean, optional
Determines whether the output is averaged in time or not. The averaged output corresponds to the standard scattering transform, while the un-averaged output skips the last convolution by \(\phi_J(t)\). This parameter may be modified after object creation. Defaults to True. Deprecated in v0.3 in favour of T and will be removed in v0.4. Replace average=False by T=0 and set T>1 or leave T=None for average=True (default).
- oversamplinginteger >= 0, optional
Controls the oversampling factor relative to the default as a power of two. Since the convolving by wavelets (or lowpass filters) and taking the modulus reduces the high-frequency content of the signal, we can subsample to save space and improve performance. However, this may reduce precision in the calculation. If this is not desirable, oversampling can be set to a large value to prevent too much subsampling. This parameter may be modified after object creation. Defaults to 0.
- vectorizeboolean, optional
Determines wheter to return a vectorized scattering transform (that is, a large array containing the output) or a dictionary (where each entry corresponds to a separate scattering coefficient). This parameter may be modified after object creation. Deprecated in favor of out_type (see below). Defaults to True.
- out_typestr, optional
The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering coefficient with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.
- Jint
The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2 ** J.
- shapeint
The length of the input signals.
- Qint
The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave).
- Tint
temporal support of low-pass filter, controlling amount of imposed time-shift invariance and maximum subsampling
- pad_leftint
The amount of padding to the left of the signal.
- pad_rightint
The amount of padding to the right of the signal.
- phi_fdictionary
A dictionary containing the lowpass filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- psi1_fdictionary
A dictionary containing all the first-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- psi2_fdictionary
A dictionary containing all the second-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.
- max_orderint
The maximum scattering order of the transform.
- averageboolean
Controls whether the output should be averaged (the standard scattering transform) or not (resulting in wavelet modulus coefficients). Note that to obtain unaveraged output, the vectorize flag must be set to False or out_type must be set to ‘list’. Deprecated in favor of T. For more details, see the documentation for scattering.
- oversamplingint
The number of powers of two to oversample the output compared to the default subsampling rate determined from the filters.
- vectorizeboolean
Controls whether the output should be vectorized into a single Tensor or collected into a dictionary. Deprecated in favor of out_type. For more details, see the documentation for scattering.
- out_typestr
Specifices the output format of the transform, which is currently one of ‘array’ or ‘list’. If ‘array’, the output is a large array containing the scattering coefficients. If ‘list’, the output is a list of dictionaries, each containing a scattering coefficient along with meta information. For more information, see the documentation for scattering.
- class kymatio.jax.Scattering2D(J, shape, L=8, max_order=2, pre_pad=False, backend='jax', out_type='array')¶
Bases:
ScatteringJax,ScatteringNumPy2DThe 2D scattering transform
The scattering transform computes two wavelet transform followed by modulus non-linearity. It can be summarized as
\(S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]\)
where
\(S_J^{(0)} x = x \star \phi_J\),
\(S_J^{(1)} x = [|x \star \psi^{(1)}_\lambda| \star \phi_J]_\lambda\), and
\(S_J^{(2)} x = [||x \star \psi^{(1)}_\lambda| \star \psi^{(2)}_\mu| \star \phi_J]_{\lambda, \mu}\).
where \(\star\) denotes the convolution (in space), \(\phi_J\) is a lowpass filter, \(\psi^{(1)}_\lambda\) is a family of bandpass filters and \(\psi^{(2)}_\mu\) is another family of bandpass filters. Only Morlet filters are used in this implementation. Convolutions are efficiently performed in the Fourier domain.
Example
# Set the parameters of the scattering transform. J = 3 M, N = 32, 32 # Generate a sample signal. x = np.random.randn(M, N) # Define a Scattering2D object. S = Scattering2D(J, (M, N)) # Calculate the scattering transform. Sx = S.scattering(x) # Equivalently, use the alias. Sx = S(x)
- Parameters:
J (int) – Log-2 of the scattering scale.
shape (tuple of ints) – Spatial support (M, N) of the input
L (int, optional) – Number of angles used for the wavelet transform. Defaults to 8.
max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.
pre_pad (boolean, optional) – Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally. Defaults to False.
backend (object, optional) – Controls the backend which is combined with the frontend.
out_type (str, optional) – The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering path with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.
- J¶
Log-2 of the scattering scale.
- Type:
int
- shape¶
Spatial support (M, N) of the input
- Type:
tuple of ints
- L¶
Number of angles used for the wavelet transform.
- Type:
int, optional
- max_order¶
The maximum order of scattering coefficients to compute. Must be either 1 or 2.
- Type:
int, optional
- pre_pad¶
Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally.
- Type:
boolean
- Psi¶
Contains the wavelets filters at all resolutions. See filter_bank.filter_bank for an exact description.
- Type:
dictionary
- Phi¶
Contains the low-pass filters at all resolutions. See filter_bank.filter_bank for an exact description.
- Type:
dictionary
- M_padded, N_padded
Spatial support of the padded input.
- Type:
int
- out_type¶
The format of the scattering output. See documentation for out_type parameter above and the documentation for scattering.
- Type:
str
Notes
The design of the filters is optimized for the value L = 8.
The pre_pad flag is particularly useful when cropping bigger images because this does not introduce border effects inherent to padding.
