Can't add a post title using speech recognition software

[afercia]

Highly recommended video (only 3 minutes) to understand how building non-native interfaces can easily break the most obvious features.
/cc @karmatosed @jasmussen @mtias @aduth @youknowriad

Adding a title to a Wordpress post using Gutenberg 2.3 and Dragon 14
https://www.youtube.com/watch?v=w7uG17NvHLQ

Thanks to Eric Wright for the excellent video and to Rian Rietveld for involving assistive technologies testers.

Gutenberg Version 2.3
Dragon 14
Chrome latest

Two issues:

1 (minor)
Saying "click Add Title" won't set my focus there
This is because the title textarea doesn't have a properly associated label. This is one of the reasons why form fields should always have visible associated labels. Annoying issue, but at least users can tab to the title.

2 (major)
when tabbing away from the title, the dictated text disappears completely. I have no idea why. Of course, this doesn't happen on a simple textarea. I can only guess there's some conflict between the events Gutenberg uses on this textarea and events triggered by Dragon.

@rianrietveld any chance to involve the Dragon team here for some feedback?

[aduth]

Very informative video. Thanks for sharing.

Saying "click Add Title" won't set my focus there

In saying "click Add Title" or otherwise using voice dictation to navigate to a field, what are the attributes that these tools are using to determine where the field is on that page? Is this something we could "fake" with an attribute of "Add Title" which is not the placeholder? A hidden label?

Aside: Should software like this be taking placeholder into consideration?

when tabbing away from the title, the dictated text disappears completely.

This looked particularly bad. Digging into the implementation of PostTitle, it's responding to the change event from a textarea field. I'm curious if this would be handled any better on an input event (which is probably the more appropriate event to be using anyways).

[aduth]

From: http://supportcontent.nuance.com/kbfiles/2626_DNS_HTML.pdf

NAVIGATION (VOICE COMMANDS)

[...]

Text Areas (<TEXTAREA>)
You can assign an ALT or TITLE attribute to a text area. As with edit controls, you need to find a
way to make the associated text apparent to the user, and the text should conform to the general
recommendations.

[afercia]

what are the attributes that these tools are using to determine where the field is on that page? Is this something we could "fake" with an attribute of "Add Title" which is not the placeholder? A hidden label?

Normally, that would be a properly associated, visible label. Software can easily check when a <label> has a for attribute, grab the value, search for an element with that ID. Or, check if a label wraps an element. This is also used to give a field a so called "accessible name".

Of course, the label needs to be visible to the user, otherwise the user wouldn't know what to say.

Even using an alt or title attribute as suggested in the note you posted above, there's the need to "find a way to make the associated text apparent to the user" before the field is focused.

That said, this is a minor issue because it is still possible to tab to the field, although that's slower for speech recognition software users.

As a last resource, Dragon can split the screen in a "grid" with 9 cells see http://nuance.custhelp.com/app/answers/detail/a_id/3430/~/information-on-the-mousegrid-command but that's a very slow process and not relevant here.

Aside: Should software like this be taking placeholder into consideration?

Apparently they don't and I doubt they will. As the accessibility team is trying to communicate, placeholders shouldn't be use as replacement for labels. For many reasons, including that in the near future placeholder attributes won't contribute any longer to the "accessible name" calculation, see https://core.trac.wordpress.org/ticket/40331#comment:3

I'm curious if this would be handled any better on an input event

I really have no idea why the text disappears, and really don't know if an input field or the planned title block would behave better.

[StommePoes]

Yup, visible labels that actually are the label are a major thing for voice users.

I could see AT taking in placeholders as part of the accessible-name algorithm-- I've seen some screen readers do it, though what they announce when sometimes depends on how the user got to the input (by tabbing in a focus mode or by arrowing in a browsing mode for example). But I wouldn't want to encourage placeholder use even if they did start taking it into consideration.

You can assign an ALT or TITLE attribute to a text area.

Alt doesn't make any sense to me. Is that a WP thing? In HTML alt's an image-only thing.

I am in general wary of onchange listeners on inputs (browsers do differ on when something counts as onchange-- for example on select elements, IE considers merely arrowing around the options in order to see them as a change, while other browsers don't), and it does sound like something to try (changing the listener from onchange to oninput, which I also like for inputs. The last thing I remember not working with oninput was IE9 wouldn't count hitting the delete key on that one). If listeners are conflicting with each other, that can be an easy way to stop that.

[afercia]

In HTML alt's an image-only thing.

Actually, that's not completely exact 😉 It can be used on other elements too, though I wouldn't consider it in this specific case.
https://www.w3.org/TR/html52/sec-forms.html#the-input-element

Worth reminding the React onchange is something a bit different from the native change event. Not sure how this could relate to text disappearing issue though.

[afercia]

Just came into my mind: as far as I can tell, Dragon should be able to get aria-label attributes, at least in its latest versions /cc @grahamarmfield

Since there's already a visible "Add title", the aria-label should give the same accessible name:
aria-label="Add title" and hopefully help.

This won't help when the title is already populated with some text, unless users remember the field name is "Add title". Note: it's a workaround, not the best example of accessible labeling 🙂

The second issue, the disappearing text, is still a mystery to me.

[aduth]

I'd be happy to put together a test branch changing the PostTitle to react to a textarea's input event instead of change, but I don't have a means to test myself whether it has any impact (no access to Dragon).

[rianrietveld]

@aduth
give me a ping when you have a PR ready, then I'll organize the Dragon troops for a test.

[aduth]

@rianrietveld I've not made a pull request for it (yet), but you can use the branch try/post-title-input (aa2ba67) to test.

Let me know if you need any help setting this up.

[akorik]

Hi Everyone,

I took a look at the video Eric Wright put together and I have a couple of comments and a workaround.
In the video Eric is tabbing down to the Title field...I know you guys are discussing using the mouse grid and that is a great Dragon feature but may I suggest instead you try using the (show links) command. Since you are using Dragon with Chrome you should have our Dragon Web Extension installed.

Example: Navigate to the post-new.php page.
Turn on your mic and say "show links" - you will see a bunch of green squares with numbers in them
With the mic still on say "choose 23" - 23 is the number on my page that correlates to the Title field.

As far as the dictated title text disappearing - That is an interesting bug. You guys already know that if you key in the title it remains (great clue). My workaround isn't elegant but it will do the job.

Prerequisite: Copy a word to your clipboard
Example: Navigate to the post-new.php page.
Turn on your mic and say "show links" - you will see a bunch of green squares with numbers in them.
With the mic still on say "choose 23" - 23 is the number on my page that correlates to the Title field.
Say "paste that" - The word from your clipboard will be pasted into the Title field.
Now say "delete that" - The word will be deleted.
Now dictate your title and say "tab key" - Your title will remain and your cursor will to the next block.

Hope this helps,
Alex

[rianrietveld]

@aduth I made an install on http://wpaccess.org/gutenberg-git/ with the try/post-title-input branch and asked the Dragon testers to have a look at adding a title to test. Will report back when they tested.

[ewaccess]

I was able to test the latest install. Dragon still did not recognize "click Add title," but it might be an issue with Dragon. I wrote some simple HTML5 pages with nothing but a <textarea> and an aria-label, and Dragon focus those either.

@akorik, thank you for the pointer on "Show links!" I've been using "click [element]" for decades, but I had no idea "Show links" existed. I was able to set focus to the title block using that. Once I had it in my head that the basic HTML navigation commands were available in Gutenberg, I was also able to set focus to the title block using "click Edit" and "click Text."

Initially I was unsure how to set focus because Gutenberg struck me as an app, so I thought I only had the desktop commands available to me, in other words "click [accessible name]"

Unfortunately, text is still not persistent when dictating directly into the Title block. It is if I use Alex's workaround above (or the Dictation Box). Dictated text is persistent when dictating into the text block below the Title block.

Thank you for your work on this!

[StommePoes]

I've not seen anywhere in Dragon command lists "show links" nor would I think of an input as a link. Is this more like "Show Numbers" in Windows Speech Rec (shows all "clickables" regardless of type)?

So while there's now a type of solution, it's not really since then Gutenberg would need to include such instructions to speech users.

I wrote some simple HTML5 pages with nothing but a <textarea> and an aria-label, and Dragon focus those either.

Eric, is that it did or did not focus?

[ewaccess]

@StommePoes Yikes, sorry for the critical omission there. That's did not focus. I know Dragon recognizes aria-label on some inputs, but apparently not on textarea.

[afercia]

@ewaccess thanks so much for all your testing and feedback! And the excellent videos :) Unfortunately I can't test with Dragon :( When you have a chance, would you mind trying to add a role="textbox" to the textarea in your HTML test to see if the aria-label is recognized?

[ewaccess]

Good idea, but unfortunately Dragon doesn't recognize it. Here's a simple pen: https://codepen.io/ewaccess/pen/zWrPeq

To confirm, I do have "show links" and "click Edit" at my disposal to set focus the title block. Neither occurred to me at the time because I was thrown by the app-like nature of Gutenberg and thought I could only say a name.

[afercia]

Thanks @ewaccess. Hm... new attempt: with a visually hidden <label> element, see this pen forked from your one: https://codepen.io/afercia/pen/LdGXQR

[ewaccess]

@afercia success! The visually hidden label works. Confirmed with Dragon 14 and 15 (latest version).

[afercia]

Yay! 🎉 So, aria-label doesn't work at all on textarea elements with Dragon. I'd be curious to know if there's some documentation about Dragon support for ARIA. Maybe @grahamarmfield ?

[StommePoes]

if there's not, perhaps Eric and I could make such documentation? Eric's shown several other issues that developers don't tend to know about (and can't do so much about either) such as the problem with using input type=number.

Edit I don't mean to sign Eric up for volunteer work :P But I'm very much a Dragon newb, I mostly only use it in IE to play with web pages.

[ewaccess]

@StommePoes that sounds like a great idea. Happy to document Dragon accessibility support with you. The more the merrier!

[afercia]

Re: the "click Add title" issue, there's now a PR to use a visually hidden <label> for the post title: #5669

Re: the disappearing dictated text, I've prepared two pens to quickly test what happens using onInput instead of onChange. While in React these are basically equivalent, the event name the browser exposes to assistive technologies could play a role. It's' just a guess, but worth testing.

Textarea:
https://codepen.io/afercia/full/EEgGWY/

Search field (to cover the case of the insert link issue):
https://codepen.io/afercia/full/QmKZeE/

Some testing with Dragon would be greatly appreciated :)

[ewaccess]

All inputs successful! Each field has Full Text Control. I was able to dictate text into and manipulate text within all four inputs. Text was persistent when tabbing and shift tabbing into the fields.

[grahamarmfield]

Hi Andrea, and Everyone. I have tested the two Codepens in both IE11 and Chrome 64 on a Windows 10 machine, with Dragon NaturallySpeaking v14. On both browsers, and in both Codepens, I was able to enter dictate text into both fields. It was harder to reference the field with the label of onChange directly but onInput usually could be referenced directly. Voicing "Press tab" and "Press shift tab" allowed me to move between the fields. BUT, neither dictating the text, nor using voice commands to tab around seemed to fire either the onchange or input event handlers. Interestingly, issuing the "Backspace" command did fire both the events. Couldn't test with Firefox as there currently isn't a Dragon plugin compatible with Firefox 58. Hope this helps. Regards Graham Armfield coolfields.co.uk <http://www.coolfields.co.uk/> M:07905 590026 T: 01483 856613 @coolfields <https://twitter.com/coolfields>

…

On Sat, Mar 17, 2018 at 7:24 PM, Eric Wright ***@***.***> wrote: All inputs successful! Each field has Full Text Control. I was able to dictate text into and manipulate text within all four inputs. Text was persistent when tabbing and shift tabbing into the fields. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5468 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACclgeL7M2gmkFpqu-giI6GC1k7M5UQgks5tfWL6gaJpZM4SgT6Q> .

[afercia]

@ewaccess @grahamarmfield thanks so much for testing. Sorry for the onChange / onInput labels, I've used them just for testing purposes.

neither dictating the text, nor using voice commands to tab around seemed to fire either the onchange or input event handlers

Hm do you mean the text below the fields didn't update? That's weird. Makes me wonder which event gets fired when Dragon populates the field with the dictated text.

[afercia]

While testing, I've noticed some inconsistencies in the way the Undo/Redo commands work. I'm going to open a separate issue for this. One possible reason for the disappearing dictated text here is that, for some reason, using Dragon triggers the "Undo" in Gutenberg so the text gets removed.

One way to test this would be trying to dictate text in a post title that already contains some text:

• create a post, add a title and a short text in a paragraph without using Dragon
• publish
• navigate away
• reopen the post
• using Dragon, add some text to the existing post title
• tab away with Dragon and see what happens

[ewaccess]

Hi @afercia, I just followed the steps above (Dragon 15/Chrome 65) and recorded a 2-minute video. https://www.youtube.com/watch?v=VToDhQOQdSQ

Result: The dictated text disappears when I tab away. The typed text remains. Anecdotally, this supports what I saw earlier with the Ctrl+Z shortcut not working as expected.

[afercia]

Thanks again @ewaccess for your testing :) That's very interesting and it seems to me we can now exclude some kind of bug that "empties" the title content.
Instead, seems to me what's happening here is that, for some reason, tabbing away while using Dragon triggers the Gutenberg "Undo" command so only the last added text gets removed. Now we have just to understand why :)

On a side note, can I publish somewhere the full 48 minutes video of your testing session? I'd really like to make it available to all the people in the team.

[afercia]

For reference:

What we learned from getting our autocomplete tested for accessibility
https://accessibility.blog.gov.uk/2018/05/15/what-we-learned-from-getting-our-autocomplete-tested-for-accessibility/

When she dictated ‘France’, the text would appear as normal. When she tried to continue, the field would go blank.
Later we found out this was because Dragon uses a special way to input the text, and autocomplete wasn’t expecting this.

Maybe someone at gov.uk knows the answer.

[afercia]

I've tweeted about the "disappearing" text and got a helpful hint from the UK gov digital service team, see https://twitter.com/tvararu/status/999758793276829700 They faced a very similar issue for their autocompleter implementation and ended up with a solution where they "poll" the input field. See https://github.com/alphagov/accessible-autocomplete/pull/15/files

@aduth when you have a chance: does it shed some light on what's going on with the disappearing text?

[aduth]

To be honest, polling is an imperfect solution, and isn't so much a compatibility solution as it is abandoning there being any possibility of interpreting the change intent from Dragon directly†.

The costs of polling are:

• Performance impact of code running on an interval. This should be limited as much as possible.
• Changes are not reflected immediately, which could manifest itself in various negative experiences:
  • Flickering UI as changes take effect
  • Race conditions between the user's intended input and the polled interpretation of the value

Based on the previous-mentioned Tweet describing Dragon's behavior of modifying the value property, some options we could potentially explore include:

• Intercept setting of HTMLInputElement.value via Object.defineProperty
  • Caveat: Unsure if this is possible for DOM interfaces, or if it's frozen/blocked from manipulation
  • This is similar to how Vue tracks property changes in 2.x
• Intercept setting of HTMLInputElement.value via Proxy
  • Caveat: Requires a browser which supports Proxy (not Internet Explorer). It cannot be polyfilled.
  • This is similar to how Vue is expected to track property changes in 3.x

† For all I know, there is no way to interpret this change directly, in which case polling may be the only option. And it is certainly a short-term resolution, but one which we should acknowledge comes at the cost of future maintainability. Which is to say: I don't mean to slander with the phrase "imperfect"; sometimes an imperfect solution is the correct solution.

[tvararu]

@aduth that's a solid writeup! I don't recall us considering Object.defineProperty, that definitely seems worth exploring as it's compatible back to IE9. 👍

[aduth]

Another idea I considered which unfortunately won't work: If value was a reflective property, meaning it sets the equivalent value attribute when changed, then a MutationObserver object observing with an attributes: true options value could detect the change in value attribute. However, from what I can tell, value property changes do not reflect back to the attribute.

[aduth]

Object.defineProperty looks promising:

• Without fix: https://codepen.io/aduth/pen/ZREvXX
• With fix: https://codepen.io/aduth/pen/XYWVeZ

[aduth]

I guess it should be stated: Dragon is partly at fault here with setting the property explicitly, since it doesn't trigger any corresponding events, requiring us to resort to such workarounds. Ideally they'd set the value in a fashion more in line with default browser behaviors (where events would be triggered).

[aduth]

I don't recall us considering Object.defineProperty, that definitely seems worth exploring as it's compatible back to IE9. 👍

To the point of compatibility, it should only need to go back so far as you intend to support a browser with Dragon dictation, if it's assumed you can rely on input / change events for all other usage.

[afercia]

I guess it should be stated: Dragon is partly at fault here

Welcome to my world, where fighting with assistive technologies is like fighting with the browsers we used to have 15 years ago 😬That's one of the reasons I always try to recommend to keep things as simple and "standard" as possible. Dragon has clear responsibilities in what's going on here. On the other hand, maybe the post title textarea is going a bit beyond what can be considered "standard".

Aside: why the isSelected state is set to false at any keypress event?

Thanks again to @tvararu @aduggin and all at the gov.uk digital service. (Love open source ❤️)

[annezazu]

Just looping back on this open issue to close it out in light of this PR being merged: #5669 If this is incorrect, feel free to re-open of course :)