Benchmarking resnet50 on tensorflow with cpu as backend and iree with cpu as backend

[HemaSowjanyaMamidi]

Hi All,
@powderluv, @benvanik, @stellaraccident
I am new to this IREE compiler so I tried checking out the inference speeds on this compiler and with TF on the CPU. I observed that TF on the CPU runs faster than the IREE compiler. Can anyone tell the reason why the inference is slow?
Any resources where I can get the information on why IREE is running slower could be really helpful.
Can optimizations be done to the code to make it run faster? If so can you please guide me.
Code: https://github.com/google/iree/blob/main/colab/resnet.ipynb
I used the same code from the IREE Github.

I sharing the table here.

Model	CPU/GPU	Baseline Accuracy	Inference time for a single image (Keras)	Inference time for batch (Keras)	Accuracy_IREE	Inference time for a single image(IREE)	Inference time for batch (IREE)
Resnet 50	CPU	82	0.171	8.56	82	0.22	11.18
Resnet 50	GPU	82	0.046	2.33	82	2.28	114.25

Thanks,
Hema

[powderluv]

Please try with these flags: https://github.com/powderluv/transformer-benchmarks/blob/e6abc1304956a059d530ce8eea29a01d491d73e8/benchmark.py#L295

[HemaSowjanyaMamidi]

I didn't understand to which function I should pass those flags in that code..Can you please help me out
I am using Tensorflow not Pytorch

[benvanik]

How are you measuring this? What are the units?
In general the only tool I trust is iree-benchmark-module, though @powderluv has some higher level tools they've been using.