AREX issues (and what is so special about lxplus?)

[marianne013]

So I was trying to see if I could get to the bottom on these AREX issues.
Misusing a very recently installed v7.3.33 UI I get:
source ../check_test_dirac/2022_Nov_17_1245_lz_py3/diracos/diracosrc
(base) lx03:arex > dirac-proxy-init -g gridpp_user
Generating proxy...
Enter Certificate password: **********
Added VOMS attribute /gridpp
[...]
VOMS fqan : ['/gridpp']
[...]

Now I send a job to UKI-SOUTHGRID-RALPP by hand:

(base) lx03:arex > arcsub --computing-element heplnx206.pp.rl.ac.uk:60000 --submission-endpoint-type org.nordugrid.arcrest arc_test.rsl
Job submitted with jobid: https://heplnx206.pp.rl.ac.uk:60000/arex/rest/1.0/jobs/Jj1KDmQKuF2n8FVDjqQVj3jqav4PzpABFKDmzEvODmrdGKDmduhmgm

(base) lx03:arex > arcstat https://heplnx206.pp.rl.ac.uk:60000/arex/rest/1.0/jobs/Jj1KDmQKuF2n8FVDjqQVj3jqav4PzpABFKDmzEvODmrdGKDmduhmgm
Job: https://heplnx206.pp.rl.ac.uk:60000/arex/rest/1.0/jobs/Jj1KDmQKuF2n8FVDjqQVj3jqav4PzpABFKDmzEvODmrdGKDmduhmgm
State: Finished
Exit Code: 0

Status of 1 jobs was queried, 1 jobs returned information
I try with my 8.0 UI, installed in October, also works.

Now I am wondering if there is something in v8.1 that doesn't quite work (though I can't see how, it should just be the diracos version that counts), so I log into lxplus, install a UI from the last hackathon and what do I get:
(base) lxplus766:d81 :~] arcsub --computing-element heplnx207.pp.rl.ac.uk:60000 --submission-endpoint-type org.nordugrid.arcrest arc_test.rsl
ERROR: Job submission aborted because no resource returned any information

On the working submission the debug output gives:

DEBUG: > GET https://heplnx206.pp.rl.ac.uk:60000/arex/rest/1.0/info HTTP/1.1
Host: heplnx206.pp.rl.ac.uk:60000
Connection: keep-alive
accept: text/xml
user-agent: ARC


DEBUG: No security processing/check requested for 'incoming'
DEBUG: < HTTP/1.1 200 OK
DEBUG: < Content-Length: 186159
DEBUG: < Connection: keep-alive
DEBUG: < content-type: application/xml
DEBUG: < strict-transport-security: max-age=31536000; includeSubDomains
VERBOSE: CONTENT 186159: <InfoRoot>
  <Domains xmlns="http://schemas.ogf.org/glue/2009/03/spec_2.0_r1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://raw.github.com/OGF-GLUE/XSD/master/schema/GLUE2.xsd">
    <AdminDomain BaseType="Domain" CreationTime="2022-11-17T14:07:39Z" Validity="10800">
      <ID>urn:ad:UKI-SOUTHGRID-RALPP</ID>

The 'glue' bit makes me think about a bdii, except I haven't set one:
(base) lx02:arex > env | grep -i bdii
(base) lx02:arex > ...crickets....

The failing one on lxplus reads at the same point:

DEBUG: > GET https://heplnx206.pp.rl.ac.uk:60000/arex/rest/1.0/info HTTP/1.1
Host: heplnx206.pp.rl.ac.uk:60000
Connection: keep-alive
accept: text/xml
user-agent: ARC


DEBUG: No security processing/check requested for 'incoming'
DEBUG: < HTTP/1.1 403 User can't be assigned configuration
DEBUG: < Content-Length: 0
DEBUG: < Connection: keep-alive
DEBUG: < strict-transport-security: max-age=31536000; includeSubDomains
DEBUG: Setting status (FAILED) for endpoint: heplnx206.pp.rl.ac.uk:60000 (org.nordugrid.arcrest, capabilities: information.discovery.resource)
DEBUG: Checking for suspended endpoints which should be started.
DEBUG:   Status of endpoint (heplnx206.pp.rl.ac.uk:60000 (org.nordugrid.arcrest, capabilities: information.discovery.resource)) is FAILED
DEBUG: Record about new job successfully added to the database (/afs/cern.ch/user/d/dbauer/.arc/jobs.dat)
ERROR: Job submission aborted because no resource returned any information

I feel like I should know what DEBUG: < HTTP/1.1 403 User can't be assigned configuration is trying to tell me, but I don't.

I've stared at the environment on both systems, but to no avail. Anyone has any idea what is happening ?

tldr; Some variation of AREX at RALPP seems to be working (now?), though not through DIRAC and not on lxplus, though probably for different reasons.

[marianne013]

AREX now works in DIRAC, I still don't understand why lxplus is so special, but I admit to it not being a DIRAC problem.