Lookup: Normalize Unicode form before string comparison to support NFD input (T1326069)

[r-farkhutdinov]

No description provided.

[Copilot]

Pull request overview

This PR addresses Lookup search mismatches when user input is in decomposed Unicode form (NFD) while data items are stored in composed form (NFC), ensuring consistent string comparisons during case-insensitive search.

Changes:

• Normalize strings to NFC in toComparable (case-insensitive path) to make NFD/NFC-equivalent strings comparable.
• Add a QUnit regression test verifying Lookup search finds NFC items when the search input is provided in NFD form (T1326069).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
packages/devextreme/testing/tests/DevExpress.ui.widgets.editors/lookup.tests.js	Adds a regression test for NFD (input) vs NFC (items) search behavior in Lookup.
packages/devextreme/js/__internal/core/utils/m_data.ts	Normalizes strings to NFC in toComparable before case-folding to support canonical-equivalent Unicode comparisons.

[Copilot]

Pull request overview

This PR updates DevExtreme’s string-comparison normalization to better handle Unicode canonical equivalence (NFD vs NFC) during lookup search, and adds a regression test to ensure NFD search text matches NFC data items (T1326069).

Changes:

• Normalize strings to NFC in toComparable for case-insensitive comparison paths.
• Add a QUnit test verifying Lookup search finds NFC items when the user enters an NFD-equivalent query.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
packages/devextreme/testing/tests/DevExpress.ui.widgets.editors/lookup.tests.js	Adds a regression test covering NFD search input vs NFC item text.
packages/devextreme/js/__internal/core/utils/m_data.ts	Normalizes strings to NFC in toComparable for case-insensitive comparisons to make matching robust to NFD input.

Comments suppressed due to low confidence (1)

packages/devextreme/js/__internal/core/utils/m_data.ts:223

• toComparable now normalizes strings to NFC only when !isCaseSensitive. Canonically equivalent strings (NFD vs NFC) will still compare as different when caseSensitive === true (e.g., case-sensitive search/highlighting paths that call toComparable with a truthy flag). If the goal is to make string comparisons Unicode-normalization-safe, consider normalizing to NFC for all string values (or at least before any comparison logic), independent of case sensitivity.

  if (typeof value === 'string' && !isCaseSensitive) {
    value = value.normalize('NFC');
    const locale = options?.locale?.toLowerCase();
    const useUpperCase = locale && !!['hy', 'el'].find((code) => locale === code || locale.startsWith(`${code}-`));

    return (useUpperCase ? toUpperCase : toLowerCase)(value, options);
  }

[Copilot]

Pull request overview

This PR improves Lookup (and other DataSource-backed text search/filter scenarios) by normalizing Unicode strings to NFC before comparison, so user input in NFD form can match data stored in NFC form (T1326069).

Changes:

• Added a QUnit regression test covering NFD search input vs NFC items for dxLookup.
• Updated toComparable to normalize case-insensitive string values to NFC prior to case folding/comparison.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
packages/devextreme/testing/tests/DevExpress.ui.widgets.editors/lookup.tests.js	Adds a regression test ensuring Lookup search matches NFC items when the query is in NFD.
packages/devextreme/js/__internal/core/utils/m_data.ts	Normalizes strings to NFC in toComparable to make comparisons robust to Unicode normalization form differences.