Problems with using Multi-Line Text fields for HTML in Sitecore

In this post I will describe a bug we fixed in Sitecore 10.3 yesterday. The bug affects the Experience Editor and prevent certain combinations for HTML tags (<link>, <a>, <img> and <image>) from being saved correctly in a Multi-Line Text field.

To understand the bug, lets first look at the module where this bug appeared: On a solution we are working on, we have a CodeSnippet module that allows the editor to insert raw HTML on a page.

The CodeSnippet module is used to include external scripts, stylesheets, and other assets on the page, both in the head and in the body. For the sake of clarity I have adjusted the module a bit, so it output the HTML in a <pre> tag on the screenshot below – but the actual module simply insert the HTML on the page where the module is inserted. So, to access and editor the HTML in the Experience Editor, we use a Custom Experience Editor Button:

However, it turned out that certain combinations of tags were truncated when saving the page. E.g. when inserting the following code snippet:

<link rel="stylesheet" href="https://www.test.com/styles.css" type="text/css"/>
<a href="https://www.test.com/index.html">Click here</a>

The code snippet would be truncated, removing the second line and saving only the link tag.

The `CallServerSavePipeline` processor

We located the problem in the ExperienceEditor.Save.CallServerSavePipeline processor, which is called before the SaveUI pipeline saves the item. The processor creates the pipeline args using the GetSaveArgs method on the PageContext. However, before the args (containing e.g. the field values to be saved) are returned, the new values are run through this peculiar filter:

private static IEnumerable<PageEditorField> FilterPageEditorFieldValue(
  List<PageEditorField> editorFields)
{
  foreach (PageEditorField editorField in editorFields)
  {
	if (editorField.Value.Contains(Constants.HtmlControlTag.Image) && 
	editorField.Value.Contains(Constants.HtmlControlTag.Img) || 
	editorField.Value.Contains(Constants.HtmlControlTag.Link) && 
	editorField.Value.Contains(Constants.HtmlControlTag.Hyperlink))
	{
	  string str = Regex.Split(
		editorField.Value, 
		Constants.HtmlControlTag.Img + 
		"|" + 
		Constants.HtmlControlTag.Hyperlink)[0];
	  if (!string.IsNullOrEmpty(str))
		editorField.Value = str;
	}
  }
  return editorFields;
}

The logic here is that if a new field value contains either "<image" and "<img" or "<link" and "<a", the new field value are truncated before the first <img or <a. And this was exactly what we saw (and the bug therefore actually therefore affected other tags like e.g. <animate>).

But what should we do?

The problem with fixing this bug is that we honestly did not understand why this filter was there in the first place? I see several options: Maybe the filtering is applied to remove some additional HTML which might be added to the field value when edited via the Experience Editor? But it could also somehow be related to the handling Sitecore IDs within e.g. a Rich Text field when using internal links and/or image tags (the actual URLs are resolved upon rendering the field, but the selection of tags could indicate a connection).

No matter what, we did not feel comfortable simply removing the filter. So, we considered a number of options, e.g. excluding a list of specific fields or certain types of fields.

In the specific solution we had a custom “code” field type which we used on the CodeSnippet module. The field is simply a Multi-Line Text field but is shown with a monotype font in the Content Editor. This also meant thatavoiding the filtering on this field type turned out to be the best approach (and probably the one I am going to recommend).

But turning off the filter off for Multi-Line Texts are certainly also an option, and the one I will show how to do in the patch below:

The patch

The patched processor is very similar to the build-in Sitecore processor, but implements a custom GetSaveArgs method instead of the method that is built into the PageContext. This method is identical to the Sitecore version except that it creates a dictionary of the saved fields type keys (which is not part of the PageEditorField model).

The dictionary is then passed on to a custom version of the FilterPageEditorFieldValue method which is now able to skipping filtering for certain field types (or potentially do the filtering in another way).

Unfortunately, the original processor contains a number of private methods, so some copying of code was needed. I have marked the parts of the processor that differs from the default Sitecore implementation with comments to make it clear where the patch changes the behaviour of Sitecore:

namespace Solution.Pipelines
{
    using Sitecore.Caching;
    using Sitecore.Data;
    using Sitecore.ExperienceEditor.Speak;
    using Sitecore.ExperienceEditor.Speak.Ribbon.Requests.SaveItem;
    using Sitecore.ExperienceEditor.Speak.Server.Contexts;
    using Sitecore.ExperienceEditor.Speak.Server.Responses;
    using Sitecore.ExperienceEditor.Switchers;
    using Sitecore.ExperienceEditor.Utils;
    using Sitecore.Globalization;
    using Sitecore.Pipelines;
    using Sitecore.Pipelines.Save;
    using Sitecore.Shell.Applications.WebEdit.Commands;
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text.RegularExpressions;

    public class CustomCallServerSavePipeline : CallServerSavePipeline
    {
        private static readonly string[] unfilteredFields = 
            new string[] { "multi-line text" };

        public override PipelineProcessorResponseValue ProcessRequest()
        {
            var value = new PipelineProcessorResponseValue();
            var pipeline = PipelineFactory.GetPipeline("saveUI");
            pipeline.ID = ShortID.Encode(ID.NewID);
            
            // Custom code: Use custom GetSaveArgs method 
            var saveArgs = CustomGetSaveArgs(RequestContext);

            var switcher = new ClientDatabaseSwitcher(RequestContext.Item.Database);

            try
            {
                pipeline.Start(saveArgs);
                CacheManager.GetItemCache(RequestContext.Item.Database).Clear();
                value.AbortMessage = Translate.Text(saveArgs.Error);
                var saveItem = GetContextSaveItem(saveArgs, RequestContext.Item.ID);
                if (saveItem != null && saveItem.Version != null)
                {
                    value.Value = saveItem.Version.Number;
                }

                return value;
            }
            finally
            {
                ((IDisposable)switcher)?.Dispose();
            }
        }

        private static SaveArgs CustomGetSaveArgs(PageContext pageContext)
        {
            var fields = WebUtility.GetFields(
                pageContext.Item.Database, 
                pageContext.FieldValues
            );

            // Custom code: Get a dictionary of all type keys
            var database = pageContext.Item.Database;

            var uniqueItems = fields.GroupBy(f => f.ItemID).
                Select(g => g.FirstOrDefault()).
                Select(i => i != null ? database.GetItem(i.ItemID) : null).
                Where(i => i != null);

            var uniqueFields = uniqueItems.SelectMany(i => i.Fields).
                GroupBy(field => field.ID).
                Select(fs => fs.FirstOrDefault()).
                Where(f => f != null);

            var typeKeys = uniqueFields.
                ToDictionary(f => f.ID, f =&gt; f.TypeKey);

            // Custom code: Using a custom filter that also get the list of type keys
            var filteredFields = CustomFilterPageEditorFieldValue(
                fields, 
                typeKeys
            );

            var layoutSource = pageContext.LayoutSource;
            SaveArgs saveArgs = PipelineUtil.GenerateSaveArgs(
                pageContext.Item, 
                filteredFields,
                string.Empty,
                layoutSource, 
                string.Empty, 
                WebUtility.GetCurrentLayoutFieldId().ToString()
            );

            saveArgs.HasSheerUI = false;
            ParseXml parseXml = new ParseXml();
            parseXml.Process(saveArgs);
            return saveArgs;
        }

        private static IEnumerable<PageEditorField< CustomFilterPageEditorFieldValue(
          IEnumerable<PageEditorField< fields, Dictionary<ID, string> typeKeys)
        {
            foreach (PageEditorField field in fields)
            {
                // Custom code: Skip filtering
                if (typeKeys.TryGetValue(field.FieldID, out var typeKey))
                {
                    if (unfilteredFields.Contains(typeKey))
                        continue;
                }

                if (field.Value.Contains(Constants.HtmlControlTag.Image) &&
                field.Value.Contains(Constants.HtmlControlTag.Img) ||
                field.Value.Contains(Constants.HtmlControlTag.Link) &&
                field.Value.Contains(Constants.HtmlControlTag.Hyperlink))
                {
                    string str = Regex.Split(
                      field.Value,
                      Constants.HtmlControlTag.Img +
                      "|" +
                      Constants.HtmlControlTag.Hyperlink)[0];
                    if (!string.IsNullOrEmpty(str))
                        field.Value = str;
                }
            }
            return fields;
        }
    }
}

To use the custom processor, we need a patch config file, replacing the existing CallServerSavePipeline with our new CustomCallServerSavePipeline:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" 
               xmlns:set="http://www.sitecore.net/xmlconfig/set/">
  <sitecore>
    <sitecore.experienceeditor.speak.requests>
      <request name="ExperienceEditor.Save.CallServerSavePipeline"
               set:type="Solution.Pipelines.CustomCallServerSavePipeline, Solution"/>
    </sitecore.experienceeditor.speak.requests>
  </sitecore>
</configuration>

With this patch in place, it is now possible to save any combination of tags in a Multi-Line Text. Thanks to Kasper Duong for sparring with me on this bug.

Problems with using Multi-Line Text fields for HTML in Sitecore

The CallServerSavePipeline processor

But what should we do?

The patch

The `CallServerSavePipeline` processor