Serialization

[wanglec]

This PR fixes #679.

Description of changes

Add set_state, get_state, serialize and deserialize functions in data_store.py and base_store.py

Possible influences of this PR.

We could serialize data_store objects.

Test Conducted

Tests are conducted in tests/forte/data/data_store_serialization_test.py

[codecov]

Codecov Report

Merging #680 (0841d5d) into master (181d293) will increase coverage by 0.11%.
The diff coverage is 95.51%.

@@            Coverage Diff             @@
##           master     #680      +/-   ##
==========================================
+ Coverage   81.20%   81.31%   +0.11%     
==========================================
  Files         249      250       +1     
  Lines       18914    19067     +153     
==========================================
+ Hits        15359    15505     +146     
- Misses       3555     3562       +7     

Impacted Files	Coverage Δ
tests/forte/data/data_store_test.py	95.04% <ø> (ø)
forte/data/base_store.py	76.11% <88.46%> (+5.66%)	⬆️
forte/data/data_store.py	91.53% <95.94%> (+0.87%)	⬆️
tests/forte/data/data_store_serialization_test.py	98.21% <98.21%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 181d293...0841d5d. Read the comment docs.

[hunterhector]

@mylibrar can you also take a look at this PR?

[hunterhector]

Should it start from 0 instead of 1?

…

On Wed, May 25, 2022 at 1:58 PM wanglec ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In forte/data/data_store.py <#680 (comment)>: > + self.__elements[k] = SortedList(self.__elements[k]) + for e in self.__elements[k]: + if e is not None: + if self._is_annotation(k): + # annotation-like + self._DataStore__tid_ref_dict[ + e[constants.TID_INDEX] + ] = e + else: + # non-annotation-like + # use `reset_index` to recalculate indices + type_name = e[constants.ENTRY_TYPE_INDEX] + if type_name in reset_index: + reset_index[type_name] += 1 + else: + reset_index[type_name] = 1 1. Yes I will add more docs. 2. Yes it is a counter of type. Since I am recreating the __tid_idx_dict, I need to know how many entries of a type already exist to assign the correct index. 3. Yes it should be tid. — Reply to this email directly, view it on GitHub <#680 (comment)>, or unsubscribe <https:/notifications/unsubscribe-auth/AAHYBN27LTXN64H2V5ZFFE3VLZS3FANCNFSM5QRFFGOA> . You are receiving this because you commented.Message ID: ***@***.***>

-- Hector, Liu Zhengzhong Website: https://hunterhector.github.io/ LinkedIn: http://www.linkedin.com/pub/hector-zhengzhong-liu/23/115/406