Could you please include the baseline translation in the downloadable System Test Results zip of a Custom Model? That would be very useful.

Of course the user can always get a baseline translation using a Translator API client, but given that Microsoft baseline models may change from one day to the other, one can never be sure if the translations obtained are the same as the ones with which the baseline BLEU score was calculated.

More details:

We make extensive usage of the Custom Translator portal. At the moment we have more than 100 Custom Models. When we train a custom model, we double check the BLEU values provided by Microsoft by recalculating them with our own quality tools.

For the BLEU calculation we use the Custom translation of the test set included in the downloadable system test results zip and we translate the source file with the translation API.

It would be very convenient for us if we also got the baseline translation in the downloadable zip for two reasons:

1. First, it would save us the time to prepare and run the translation tasks, which may not be negligible when we are double checking 20 or 25 custom models (which is not uncommon)

2. Second, precisely because the baseline translation may change from day to day, it would be convenient to include it in the downloadable system test results. Else, in the Custom Translator portal, for a given custom model, you are providing a baseline BLEU that is calculated with a baseline translation that is not available anywhere and cannot be reproduced (or, more precisely, it can be reproduced if the test set is translated with the translation API right after the custom model training is finished, but it cannot be guaranteed that this translation will be the same if it is generated a couple of weeks after the custom model was trained)

We use the detection api to decide whether the language needs to be translated. However the 'detect' api returns inaccurate results at times. Eg. 'hi siri', 'hi google' are detected as 'ca' instead of 'en'. This will then be translated to 'there are siri', which is totally unacceptable in our case.

Currently there is no way for user to correct the detection or put more weight/preference on a particular language. In the above example, although 'en' is in the alternative, they all have the same score of 1. We'd like to add some weight on a particular language to control the 'detect' accuracy. For example, we'd like to see 'hi siri' returns 'en' with score 1 but 'ca' with score 0.998 etc.

Another way is to allow user to customize/improve the English dictionary. Eg. we want to add a few terms/phrases in the dictionary so that those words are NOT detected as other language than 'en'.

Typically, verbs and nouns are mixed up, resulting in hilarious, yet totally unusable (business-wise) translations.

It seems that the translation engine lacks any kind grammar knowledge - it doesn't even try to make up sentences of what it translates.

While this technique might play out with germanic<->germanic languages, it certainly does not with Finnish language.. It might be unfair to compare it to Google Translate (which has no problem translating even complex sentences), but that's the consumer standard so ofc people are comparing to it. As a matter of fact, even Microsoft's translation built-in-feature in Twitter does way better job!

Are there any plans on improving this feature?

Below are minor UX improvements to boost user experience (some are nice to have):

1. Ability to choose a workspace as default (when ever you open the portal), this is helpful when you are member of multiple workspaces.

2. Add clipboard icon next to category id for easy copy/paste. At the moment, entire line of category id has anchor link (end up clicking it multiple times during id selection).

3. Send a welcome/invitation mail when someone adds you as a workspace owner.

4. In the projects page, when you Ctrl+click on any project - allow opening it in new window/tab (as it has unique url) for multi tasking.

5. Sticky filters - In project page, when a filter is applied, clicked on one of the project and when you navigate back to the projects page - allow fiter condition is still available until cleared (for large number porjects, it is getting tedious to filter again and choose the projects).

6. Allow user can change the number of items visible in the grid (page size), currently it is 10. When you have large number of projects, it is tedious to browse through multiple pages or redefining the filter each time to scope down.