Port transductive alignment model - #466
Conversation
ddaspit
left a comment
There was a problem hiding this comment.
@ddaspit reviewed 9 files and all commit messages, and made 3 comments.
Reviewable status: 9 of 10 files reviewed, 3 unresolved discussions (waiting on Enkidu93 and pmachapman).
src/SIL.Machine/Corpora/CorporaExtensions.cs line 1135 at r1 (raw file):
} public static IParallelTextCorpus WordAlign(
I think there are too many pitfalls with trying to implement this operation. Instead, we should add a similar IParallelTextCorpus extension method in SIL.Machine.Translation.Thot that trains and aligns a corpus using a Thot model. The method should accept a ThotWordAlignmentModelType parameter. You can have overloads that accept a ThotWordAlignmentModel or ThotSymmetrizedWordAlignmentModel as well. It would mimic the word_align_corpus function in Machine.py. If the passed in model is not configured to emit training alignments, then it should fall back to inductive alignment.
src/SIL.Machine.Translation.Thot/ThotWordAlignmentModel.cs line 169 at r1 (raw file):
{ CheckDisposed(); IntPtr nativeMatrix = Thot.AllocNativeMatrix(_sourceWords.Count, _targetWords.Count);
To get the size of the matrix, you can call swAlignModel_getTrainingAlignment with a null buffer.
src/SIL.Machine.Translation.Thot/ThotWordAlignmentModelTrainer.cs line 24 at r1 (raw file):
private readonly bool _isEflomal; public ThotWordAlignmentModelTrainer(
You forgot emitTrainingAlignments here.
Fixes #455.
Port of sillsdev/machine.py#327
@ddaspit I am unsure whether my implementations of
ThotSymmetrizedWordAlignmentModel.TrainingAlignmentCountandThotWordAlignmentModel.GetTrainingAlignment()are correct. I wrote them based on what I understood the underlying logic in thot to be doing, asSymmetrizedAlignmentModelis not exposed via the C API.If they are not right, I will update thot to expose the methods from
SymmetrizedAlignmentModelin the C API that are exposed in thot'smodule.ccand used by the machine.py implementation.This change is