Data conversion when packing #5647
Description
I think we need a feature in which data conversion is done while packing. This way all architectures could get lower resolution kernels to work by converting either A or B or BOTH to a desired format and then using a kernel that uses that format.
It would be useful for SBGEMM and SHGEMM in some situations in which there is no fast hardware instructions with MADD in these formats.
Flow -
Current:
pack -> convert -> kernel
Proposed new feature:
pack_convert -> kernel
Yes, it would increase memory requirements. But for hardware that has slower conversions we are wasting NxKxM converts that could be done as NxK and MxK - which is a much smaller number.
Plus we could save the repetitive conversions that is done sometimes.