Graphics

 View Only
  • 1.  datalinq v11 folder with thousand .txt files

    Posted 5 hours ago

    Hello,

    I am running XPression version 11 and I am running into an issue trying to configure a DataLinq source to read data from a large number of text files (around 6,500 files) located on a network share (UNC path).

    Context & Setup:

    • XPression Version: 11

    • File Location: \\nxtv-st1-fs01.rmc.local\DP1Share\

    • File Format: Plain text files using a hashtag (#) as a delimiter, with no headers (e.g., ACCOR-INTRADAY.txt).

    • Connection Type tried: ADODB (Microsoft.ACE.OLEDB.12.0) and Native Text DataLinq.

    The Problem:

    Inside XPression, I need to call these files dynamically based on a macro value (e.g., @id_society@-INTRADAY.txt or @id_society@-INTRADAY#txt).

    1. When using ADODB: I set up a schema.ini file in the folder to handle the # delimiter and formatting. However, DataLinq throws a path/file validation error. It completely rejects the UNC path (Failure: '\\...\' is not a valid path). If I map the network share to a drive letter (like V: or X:), XPression throws a "not a valid path" error as well, likely due to a Windows session/admin privilege conflict where the DataLinq service cannot see the mapped drive.

    2. When using Native Text DataLinq Source: Since the Text DataLinq Server configuration UI strictly requires selecting a specific physical file to validate the connection, I cannot leave it open to a directory. If I use a placeholder file in the server manager and then try to override it dynamically in the scene's DataLinq properties using the file=@id_society@-INTRADAY.txt syntax in the Table field, it fails to fetch the data.

    How can I properly configure XPression v11 to dynamically query thousands of individual text files on a network share using a specific delimiter, without hitting these path validation blocks?

    Any help or best practices for this workflow would be greatly appreciated!

    Thank you,



    ------------------------------
    Benjamin Callu
    Realisateur/Integrateur
    BFMBUSINESS
    ------------------------------


  • 2.  RE: datalinq v11 folder with thousand .txt files

    Posted 3 hours ago

    Must be a permissions issue. I was curious so I tried it and was able to get it to work fine.

    First, I setup an ADODB Datalinq as you described with this connection string: Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\\ls210d39a\share\Shared Documents\;Extended Properties="text;HDR=Yes;FMT=Delimited";

    And then created a schema.ini in the directory:

    [delete_me.csv]
    Format=Delimited(#)
    ColNameHeader=True
    CharacterSet=ANSI
    
    Col1=Name Text
    Col2=Instrument Text

    And a data file for testing (delete_me.csv):

    Name#Instrument
    Miles Davis#Trumpet
    John Coltrane#Sax
    Paul Chambers#Bass
    Philly Joe Jones#Drums
    Red Garland#Piano

    In the Xpression graphic, I created a text object for the file path. The text object is named "csv_file" and has a value of "delete_me#csv".

    Lastly, I create a text object to show the data with this datalinq:

    Column: Name
    Row: 2
    Table: @csv_file@

    And that text has the value of "John Coltrane".



    ------------------------------
    JohnCorigliano
    Senior Software Engineer
    ISC
    ------------------------------



  • 3.  RE: datalinq v11 folder with thousand .txt files

    Posted 2 hours ago

    It does sound like a permission issue. 

    You might be able to get the network share mapped to a drive letter if you do this to windows:

    In Registry Editor, locate and then click the following registry subkey:
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System

    Right-click Configuration, click New, and then click DWORD (32-bit) Value.

    Name the new registry entry as EnableLinkedConnections.

    Double-click the EnableLinkedConnections registry entry.

    In the Edit DWORD Value dialog box, type 1 in the Value data field, and then click OK.

    Exit Registry Editor, and then restart the computer.




  • 4.  RE: datalinq v11 folder with thousand .txt files

    Posted 2 hours ago

    And for future consideration, and for others reading this thread who may find it of interest. 

    In the next release of XPression, version 14.0, later this summer. The Datalinq XML Data source supports this new watch folder mode:

    As you can read in the image, this will allow Datalinq to monitor a folder for XML files (sorry Benjamin not text files) and create tables for each XML file found.