1

I'm developing a helper application that provides autocomplete features for any application by using macOS Accessibility APIs. I can get the right location from Xcode, VS Code and other applications using kAXSelectedTextRangeAttribute (they automatically select the range I guess) .

However, I cannot get it from TextEdit, Slack etc.. The selection range comes with a 0 length for them.

P.S. Necessary permissions are granted for the app.

This method works if kAXSelectedTextRangeAttribute is not 0 length:

func getCursorPosition() {
    let systemWideElement = AXUIElementCreateSystemWide()
    var focusedElement : AnyObject?
    
    let error = AXUIElementCopyAttributeValue(systemWideElement, kAXFocusedUIElementAttribute as CFString, &focusedElement)
    if (error != .success){
        print("Couldn't get the focused element. Probably a webkit application")
    } else {
        var selectedRangeValue : AnyObject?
        let selectedRangeError = AXUIElementCopyAttributeValue(focusedElement as! AXUIElement, kAXSelectedTextRangeAttribute as CFString, &selectedRangeValue)
        if (selectedRangeError == .success){
            var selectedRange : CFRange?
            AXValueGetValue(selectedRangeValue as! AXValue, AXValueType(rawValue: kAXValueCFRangeType)!, &selectedRange)
            var selectRect = CGRect()
            var selectBounds : AnyObject?
            let selectedBoundsError = AXUIElementCopyParameterizedAttributeValue(focusedElement as! AXUIElement, kAXBoundsForRangeParameterizedAttribute as CFString, selectedRangeValue!, &selectBounds)
            if (selectedBoundsError == .success){
                AXValueGetValue(selectBounds as! AXValue, .cgRect, &selectRect)
                    print("CursorX \(selectRect.origin.x), CursorY: \(selectRect.origin.y)")
            }
        }
    }
}

However below method doesn't give the right location, tried to follow another approach for TextEdit, Slack etc:

AXUIElement extended to access properties by extension methods in the below code (position, size and insertionPointLineNumber)

func getCursorPositionHacky(element: AXUIElement) {
    if element.position != nil {
        if element.size != nil {
            if element.insertionPointLineNumber != nil {
                
                // HACK: Calculate the cursor's approximate screen coordinates
                
                let lineHeight: CGFloat = 14.0 // This is an estimation from Inspector view
               
                var cursorX = element.position!.x + 10.0
 
              
                let cursorY = element.position!.y + (CGFloat(element.insertionPointLineNumber! - 1) * lineHeight)
                print("Cursor position is approximately at: (\(cursorX), \(cursorY))")
            }
        }
    }
}

Any help would be appreciated 🙏🏼

5
  • Duplicate of How to get focused element's cursor position using accessibility API in macOS?. Can you get the location of the character before or after the insertion point?
    – Willeke
    Commented Apr 16 at 13:07
  • Related: stackoverflow.com/questions/78323684/…
    – soundflix
    Commented Apr 16 at 14:03
  • @Willeke I was able to get the cursor position for TextEdit which returns selected range 0, by creating a range manually. For Electron apps this approach did not work either .. var previousCharRange = CFRangeMake(selectedRange.location - 1, 1)
    – ziyasal
    Commented Apr 16 at 15:19
  • For VSCode, when the AXRole is AXTextArea the AXPosition attribute worked well.
    – ziyasal
    Commented Apr 16 at 15:54
  • I will update here with my full solution for various cases when I get Slack to work.
    – ziyasal
    Commented Apr 16 at 15:57

0

Browse other questions tagged or ask your own question.