Skip to content

Resource Exhausted Error while testing  #10

Description

@prathmeshrmadhu
 ResourceExhaustedError` (see above for traceback): OOM when allocating tensor with shape[1,64,1600,1330] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc

	 [[node Conv2D_10 (defined at /localhome/prathmeshmadhu/work/EFI/Code/arbitrary_style_transfer/encoder.py:95)  = Conv2D[T=DT_FLOAT, data_format="NCHW", dilations=[1, 1, 1, 1], padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](Conv2D_10-0-TransposeNHWCToNCHW-LayoutOptimizer, encoder/conv1_2/kernel/read)]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

	 [[{{node clip_by_value/_77}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_487_clip_by_value", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

I am trying to generate some 40000+ different stylized images and it generates some 220+ images and then always throws this resources error. My hardware details are as follows :

GPU : GeForce GTX 970, 4GB.
CPU : 8 cores, 64 GB.

Do let me know if you need any further details.
Thanks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions