# Image Processor Image processor负责为视觉模型准备输入特征并后期处理处理它们的输出。这包括诸如调整大小、归一化和转换为PyTorch、TensorFlow、Flax和NumPy张量等转换。它还可能包括特定于模型的后期处理，例如将logits转换为分割掩码。 ## ImageProcessingMixin[[transformers.ImageProcessingMixin]] #### transformers.ImageProcessingMixin[[transformers.ImageProcessingMixin]] [Source](https://github.com/huggingface/transformers/blob/v4.57.1/src/transformers/image_processing_base.py#L64) This is an image processor mixin used to provide saving/loading functionality for sequential and image feature extractors. from_pretrainedtransformers.ImageProcessingMixin.from_pretrainedhttps://github.com/huggingface/transformers/blob/v4.57.1/src/transformers/image_processing_base.py#L91[{"name": "pretrained_model_name_or_path", "val": ": typing.Union[str, os.PathLike]"}, {"name": "cache_dir", "val": ": typing.Union[str, os.PathLike, NoneType] = None"}, {"name": "force_download", "val": ": bool = False"}, {"name": "local_files_only", "val": ": bool = False"}, {"name": "token", "val": ": typing.Union[str, bool, NoneType] = None"}, {"name": "revision", "val": ": str = 'main'"}, {"name": "**kwargs", "val": ""}]- **pretrained_model_name_or_path** (`str` or `os.PathLike`) -- This can be either: - a string, the *model id* of a pretrained image_processor hosted inside a model repo on huggingface.co. - a path to a *directory* containing a image processor file saved using the [save_pretrained()](/docs/transformers/v4.57.1/zh/main_classes/image_processor#transformers.ImageProcessingMixin.save_pretrained) method, e.g., `./my_model_directory/`. - a path or url to a saved image processor JSON *file*, e.g., `./my_model_directory/preprocessor_config.json`. - **cache_dir** (`str` or `os.PathLike`, *optional*) -- Path to a directory in which a downloaded pretrained model image processor should be cached if the standard cache should not be used. - **force_download** (`bool`, *optional*, defaults to `False`) -- Whether or not to force to (re-)download the image processor files and override the cached versions if they exist. - **resume_download** -- Deprecated and ignored. All downloads are now resumed by default when possible. Will be removed in v5 of Transformers. - **proxies** (`dict[str, str]`, *optional*) -- A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. - **token** (`str` or `bool`, *optional*) -- The token to use as HTTP bearer authorization for remote files. If `True`, or not specified, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). - **revision** (`str`, *optional*, defaults to `"main"`) -- The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. To test a pull request you made on the Hub, you can pass `revision="refs/pr/"`. - **return_unused_kwargs** (`bool`, *optional*, defaults to `False`) -- If `False`, then this function returns just the final image processor object. If `True`, then this functions returns a `Tuple(image_processor, unused_kwargs)` where *unused_kwargs* is a dictionary consisting of the key/value pairs whose keys are not image processor attributes: i.e., the part of `kwargs` which has not been used to update `image_processor` and is otherwise ignored. - **subfolder** (`str`, *optional*, defaults to `""`) -- In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. - **kwargs** (`dict[str, Any]`, *optional*) -- The values in kwargs of any keys which are image processor attributes will be used to override the loaded values. Behavior concerning key/value pairs whose keys are *not* image processor attributes is controlled by the `return_unused_kwargs` keyword parameter.0A image processor of type [ImageProcessingMixin](/docs/transformers/v4.57.1/zh/main_classes/image_processor#transformers.ImageProcessingMixin). Instantiate a type of [ImageProcessingMixin](/docs/transformers/v4.57.1/zh/main_classes/image_processor#transformers.ImageProcessingMixin) from an image processor. Examples: ```python # We can't instantiate directly the base class *ImageProcessingMixin* so let's show the examples on a # derived class: *CLIPImageProcessor* image_processor = CLIPImageProcessor.from_pretrained( "openai/clip-vit-base-patch32" ) # Download image_processing_config from huggingface.co and cache. image_processor = CLIPImageProcessor.from_pretrained( "./test/saved_model/" ) # E.g. image processor (or model) was saved using *save_pretrained('./test/saved_model/')* image_processor = CLIPImageProcessor.from_pretrained("./test/saved_model/preprocessor_config.json") image_processor = CLIPImageProcessor.from_pretrained( "openai/clip-vit-base-patch32", do_normalize=False, foo=False ) assert image_processor.do_normalize is False image_processor, unused_kwargs = CLIPImageProcessor.from_pretrained( "openai/clip-vit-base-patch32", do_normalize=False, foo=False, return_unused_kwargs=True ) assert image_processor.do_normalize is False assert unused_kwargs == {"foo": False} ``` **Parameters:** pretrained_model_name_or_path (`str` or `os.PathLike`) : This can be either: - a string, the *model id* of a pretrained image_processor hosted inside a model repo on huggingface.co. - a path to a *directory* containing a image processor file saved using the [save_pretrained()](/docs/transformers/v4.57.1/zh/main_classes/image_processor#transformers.ImageProcessingMixin.save_pretrained) method, e.g., `./my_model_directory/`. - a path or url to a saved image processor JSON *file*, e.g., `./my_model_directory/preprocessor_config.json`. cache_dir (`str` or `os.PathLike`, *optional*) : Path to a directory in which a downloaded pretrained model image processor should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`) : Whether or not to force to (re-)download the image processor files and override the cached versions if they exist. resume_download : Deprecated and ignored. All downloads are now resumed by default when possible. Will be removed in v5 of Transformers. proxies (`dict[str, str]`, *optional*) : A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or `bool`, *optional*) : The token to use as HTTP bearer authorization for remote files. If `True`, or not specified, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`) : The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. To test a pull request you made on the Hub, you can pass `revision="refs/pr/"`. return_unused_kwargs (`bool`, *optional*, defaults to `False`) : If `False`, then this function returns just the final image processor object. If `True`, then this functions returns a `Tuple(image_processor, unused_kwargs)` where *unused_kwargs* is a dictionary consisting of the key/value pairs whose keys are not image processor attributes: i.e., the part of `kwargs` which has not been used to update `image_processor` and is otherwise ignored. subfolder (`str`, *optional*, defaults to `""`) : In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. kwargs (`dict[str, Any]`, *optional*) : The values in kwargs of any keys which are image processor attributes will be used to override the loaded values. Behavior concerning key/value pairs whose keys are *not* image processor attributes is controlled by the `return_unused_kwargs` keyword parameter. **Returns:** A image processor of type [ImageProcessingMixin](/docs/transformers/v4.57.1/zh/main_classes/image_processor#transformers.ImageProcessingMixin). #### save_pretrained[[transformers.ImageProcessingMixin.save_pretrained]] [Source](https://github.com/huggingface/transformers/blob/v4.57.1/src/transformers/image_processing_base.py#L205) Save an image processor object to the directory `save_directory`, so that it can be re-loaded using the [from_pretrained()](/docs/transformers/v4.57.1/zh/main_classes/image_processor#transformers.ImageProcessingMixin.from_pretrained) class method. **Parameters:** save_directory (`str` or `os.PathLike`) : Directory where the image processor JSON file will be saved (will be created if it does not exist). push_to_hub (`bool`, *optional*, defaults to `False`) : Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace). kwargs (`dict[str, Any]`, *optional*) : Additional key word arguments passed along to the [push_to_hub()](/docs/transformers/v4.57.1/zh/main_classes/model#transformers.utils.PushToHubMixin.push_to_hub) method. ## BatchFeature[[transformers.BatchFeature]] #### transformers.BatchFeature[[transformers.BatchFeature]] [Source](https://github.com/huggingface/transformers/blob/v4.57.1/src/transformers/feature_extraction_utils.py#L62) Holds the output of the [pad()](/docs/transformers/v4.57.1/zh/main_classes/feature_extractor#transformers.SequenceFeatureExtractor.pad) and feature extractor specific `__call__` methods. This class is derived from a python dictionary and can be used as a dictionary. convert_to_tensorstransformers.BatchFeature.convert_to_tensorshttps://github.com/huggingface/transformers/blob/v4.57.1/src/transformers/feature_extraction_utils.py#L171[{"name": "tensor_type", "val": ": typing.Union[str, transformers.utils.generic.TensorType, NoneType] = None"}]- **tensor_type** (`str` or [TensorType](/docs/transformers/v4.57.1/zh/internal/file_utils#transformers.TensorType), *optional*) -- The type of tensors to use. If `str`, should be one of the values of the enum [TensorType](/docs/transformers/v4.57.1/zh/internal/file_utils#transformers.TensorType). If `None`, no modification is done.0 Convert the inner content to tensors. **Parameters:** data (`dict`, *optional*) : Dictionary of lists/arrays/tensors returned by the __call__/pad methods ('input_values', 'attention_mask', etc.). tensor_type (`Union[None, str, TensorType]`, *optional*) : You can give a tensor_type here to convert the lists of integers in PyTorch/TensorFlow/Numpy Tensors at initialization. #### to[[transformers.BatchFeature.to]] [Source](https://github.com/huggingface/transformers/blob/v4.57.1/src/transformers/feature_extraction_utils.py#L202) Send all values to device by calling `v.to(*args, **kwargs)` (PyTorch only). This should support casting in different `dtypes` and sending the `BatchFeature` to a different `device`. **Parameters:** args (`Tuple`) : Will be passed to the `to(...)` function of the tensors. kwargs (`Dict`, *optional*) : Will be passed to the `to(...)` function of the tensors. To enable asynchronous data transfer, set the `non_blocking` flag in `kwargs` (defaults to `False`). **Returns:** `[BatchFeature](/docs/transformers/v4.57.1/zh/main_classes/image_processor#transformers.BatchFeature)` The same instance after modification. ## BaseImageProcessor[[transformers.BaseImageProcessor]] #### transformers.BaseImageProcessor[[transformers.BaseImageProcessor]] [Source](https://github.com/huggingface/transformers/blob/v4.57.1/src/transformers/image_processing_utils.py#L38) center_croptransformers.BaseImageProcessor.center_crophttps://github.com/huggingface/transformers/blob/v4.57.1/src/transformers/image_processing_utils.py#L125[{"name": "image", "val": ": ndarray"}, {"name": "size", "val": ": dict"}, {"name": "data_format", "val": ": typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None"}, {"name": "input_data_format", "val": ": typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None"}, {"name": "**kwargs", "val": ""}]- **image** (`np.ndarray`) -- Image to center crop. - **size** (`dict[str, int]`) -- Size of the output image. - **data_format** (`str` or `ChannelDimension`, *optional*) -- The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - **input_data_format** (`ChannelDimension` or `str`, *optional*) -- The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format.0 Center crop an image to `(size["height"], size["width"])`. If the input size is smaller than `crop_size` along any edge, the image is padded with 0's and then center cropped. **Parameters:** image (`np.ndarray`) : Image to center crop. size (`dict[str, int]`) : Size of the output image. data_format (`str` or `ChannelDimension`, *optional*) : The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. input_data_format (`ChannelDimension` or `str`, *optional*) : The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. #### normalize[[transformers.BaseImageProcessor.normalize]] [Source](https://github.com/huggingface/transformers/blob/v4.57.1/src/transformers/image_processing_utils.py#L88) Normalize an image. image = (image - image_mean) / image_std. **Parameters:** image (`np.ndarray`) : Image to normalize. mean (`float` or `Iterable[float]`) : Image mean to use for normalization. std (`float` or `Iterable[float]`) : Image standard deviation to use for normalization. data_format (`str` or `ChannelDimension`, *optional*) : The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. input_data_format (`ChannelDimension` or `str`, *optional*) : The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. **Returns:** ``np.ndarray`` The normalized image. #### rescale[[transformers.BaseImageProcessor.rescale]] [Source](https://github.com/huggingface/transformers/blob/v4.57.1/src/transformers/image_processing_utils.py#L56) Rescale an image by a scale factor. image = image * scale. **Parameters:** image (`np.ndarray`) : Image to rescale. scale (`float`) : The scaling factor to rescale pixel values by. data_format (`str` or `ChannelDimension`, *optional*) : The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. input_data_format (`ChannelDimension` or `str`, *optional*) : The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. **Returns:** ``np.ndarray`` The rescaled image.