-
Notifications
You must be signed in to change notification settings - Fork 157
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft: [onert] Support RoPE operation #14090
base: master
Are you sure you want to change the base?
Conversation
f3f19e3
to
c43dfa6
Compare
This commit supports RoPE for onerte ONE-DCO-1.0-Signed-off-by: youngsik kim <ys44.kim@samsung.com>
- check data type of sin/cos input ONE-DCO-1.0-Signed-off-by: youngsik kim <ys44.kim@samsung.com>
This commit adds cker test for RoPE ONE-DCO-1.0-Signed-off-by: youngsik kim <ys44.kim@samsung.com>
ad223c2
to
12a419d
Compare
This commit change assert to throw in cker rope ONE-DCO-1.0-Signed-off-by: youngsik kim <ys44.kim@samsung.com>
kGptNeox = 0, | ||
kGptJ = 1, | ||
}; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was added to support two modes (GPT-NEOX, GPT-J) of RoPE.
Currently, only GPT-NEOX is supported.
throw std::runtime_error("the dimension(3) of input and sin_table do not match"); | ||
|
||
if (input_shape.Dims(3) != cos_table_shape.Dims(3)) | ||
throw std::runtime_error("the dimension(3) of input and cos_table do not match"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if the sin/cos table and input dim(3) are different, an error occurs in the following operation
and it is processed as throw.
auto input_tensor = _tensor_reg->getPortableTensor(input_index); | ||
auto sin_tensor = _tensor_reg->getPortableTensor(sin_table); | ||
auto cos_tensor = _tensor_reg->getPortableTensor(cos_table); | ||
auto output_tensor = _tensor_reg->getPortableTensor(output_index); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The current implementation has three inputs(input, sin_table, and cos_table)
so, input the sin/cos table at the upper layer (ex.app).
This commit adds cker test for RoPE ONE-DCO-1.0-Signed-off-by: youngsik kim <ys44.kim@samsung.com>
dda779e
to
f829437
Compare
} | ||
} | ||
} | ||
else |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BUILD_TYPE = Release
before fuse
$ ./Product/x86_64-linux.release/out/bin/onert_run ~/model/rope/Net_RoPE_000.circle
===================================
MODEL_LOAD takes 0.251 ms
PREPARE takes 3.557 ms
EXECUTE takes 0.036 ms
- MEAN : 0.036 ms
- MAX : 0.036 ms
- MIN : 0.036 ms
- GEOMEAN : 0.036 ms
===================================
after fuse
$ ./Product/x86_64-linux.release/out/bin/onert_run ~/model/rope/RoPE_000.circle
===================================
MODEL_LOAD takes 0.169 ms
PREPARE takes 2.384 ms
EXECUTE takes 0.020 ms
- MEAN : 0.020 ms
- MAX : 0.020 ms
- MIN : 0.020 ms
- GEOMEAN : 0.020 ms
===================================
@glistening @hseok-oh @ragmani @zetwhite |
This commit restore the wrong code applied ONE-DCO-1.0-Signed-off-by: youngsik kim <ys44.kim@samsung.com>
This commit adds data type for RoPE ONE-DCO-1.0-Signed-off-by: youngsik kim <ys44.kim@samsung.com>
@nnfw-bot test-all |
This commit supports RoPE Operation for onert
ONE-DCO-1.0-Signed-off-by: youngsik kim ys44.kim@samsung.com