External data file

[jimmyhartington]

Hi

 

Is it possible in a template to reference a External data file which is stored on a external server?

[Dan Korn]

Sure. You can either include the full path to the file in the call to "new ExternalDatFileEx" or, as I recommend, add the path to the Search Path on the Advanced tab of the Composition Settings dialog. If you're on Mac (and not submitting the job to Producer) you'll need to first mount the remote drive, then use a path starting with "/Volumes". If you're on Windows, you can use a UNC path (\\server\share\folder\etc). Note that if you specify the Windows backslashes in JavaScript, they'll need to be doubled up since the backslash also denotes an escape in JavaScript strings.

[jimmyhartington]

Hi Dan

 

Thanks for the answer.

 

But can see I forgot to ask more precisely.

 

I would like a way to reference an external file in a template hosted on Marcom.

 

My customer has an address file (text-file) on their website, which I would like to reference.

So the customer can change the file themselves and that way the template is automatically updated.

[Dan Korn]

I would like a way to reference an external file in a template hosted on Marcom.

 

My customer has an address file (text-file) on their website, which I would like to reference.

So the customer can change the file themselves and that way the template is automatically updated.

Sure, you can use a URL starting with "http://" in the call to the ExternalDataFileEx constructor function, just like with the Resource constructor function (or CreateResource), for example:

var XDF = new ExternalDataFileEx("http://www.sample-videos.com/csv/Sample-Spreadsheet-10-rows.csv");

[jimmyhartington]

Thanks.

 

I will try this.

[kjacobson]

Is the ability to reference an external data file using a URL only available with FusionPro 10? Jimmy, were you able to get this to work on Marcom?

 

I am currently using FusionPro 9.3.9 and have been unable to make the URL work locally or on Marcom. I have no problem if the external data file is on my machine, just when it is on our webserver.

[jimmyhartington]

Hi

 

Actually I have not tried this with an external data file as text.

I think I asked the question wrong first time around. Because what I needed was to link to a folder of images on our own server.

And this works.

See this thread: http://forums.pti.com/showthread.php?t=5003

But I have thought about using a text-file as well.

 

The problem in my testing was that I used https. And apparently this is not supported in Marcom yet. But is in FusionPro 10, but maybe not in 9.

So if you code for fetching the file from your webserver uses https, then try to switch to http.

[jimmyhartington]

Hi

 

I have just tried with this code:

var xdfAdresse = new ExternalDataFileEx("http://files.skabertrang.dk/jh/test/86806344_Envidan-adresser-TEST.txt", "\t", "UTF8");
var adresse = Field("Adresse");
var recno = xdfAdresse.FindRecord("Dropdown", adresse);
var ruleadresse = xdfAdresse.GetFieldValue(recno, "Adresse");

return "<t>" + ruleadresse

 

And it does not work.

I just test it locally.

But it works if I use the file from disk.

[kjacobson]

Thank for your reply Jimmy. Your link to the other thread gave me the idea to create a resource first and it worked.

 

I added this to the OnJobStart rule:

CreateResource("http://www.yourdomain.com/filename.csv");
data = new ExternalDataFileEx("http://www.yourdomain.com/filename.csv");
return data.valid;

I also went ahead and updated to FusionPro 10, but I don't think that was the issue.

 

This worked locally and on Marcom.

[jimmyhartington]

I can confirm this also works here.

Both locally and in Marcom.

 

Why we need to create a resource of the text-file first I do not understand.

But maybe somebody from MarcomCentral has an answer.

[Dan Korn]

Sorry, apparently what I wrote back in January was not accurate. Or perhaps it was at the time, and something got broken since then.

 

The intent was indeed that URLs could be used in ExternalDataFileEx calls. Unfortunately, it seems that the code change made to the CreateResource function to support this was not also made to ExternalDataFileEx (or, again, was somehow regressed).

 

I was very clever of kjacobson to find the workaround. Why that works is kind of "Inside Baseball," but basically, FusionPro keeps a cache of paths and URLs it has already resolved with the Search Path, or by downloading, so that it doesn't have to re-search or re-download the file when that same path or URL is requested again in the job. Because CreateResource does indeed download the file and add the result to the cache, it's there when the call to ExternalDataFileEx is made, and gets found.

 

I will fix this in an upcoming 10.0 build, so that the URL gets properly resolved in the ExternalDataFileEx call, without having to call CreateResource first.

 

Again, my apologies for the bad information earlier, and kudos to kjacobson for discovering the workaround!

[kjacobson]

I recently upgraded to FusionPro 10.1 and have found that this work around no longer works. In fact, I have not been able to get a URL to work with ExternalDataFileEx. Is there something I can do to make this work again?

[Dan Korn]

I recently upgraded to FusionPro 10.1 and have found that this work around no longer works. In fact, I have not been able to get a URL to work with ExternalDataFileEx. Is there something I can do to make this work again?

It works for me in 10.1, with any of the files from this page:

https://people.sc.fsu.edu/~jburkardt/data/csv/csv.html

 

For instance:

var XDF = new ExternalDataFileEx("https://people.sc.fsu.edu/~jburkardt/data/csv/addresses.csv");
return XDF.recordCount;

Are you sure that the URL you're trying is valid? Does it open up in a web browser?

[kjacobson]

Thanks for replying, Dan. I finally had a few minutes to test this out further. I've tried a couple of links on that page in additional to my own files, but I haven't been able to get the XDF URL to work. It works just fine if the file is local.

 

I also want to mention that most of my external files are Google sheets, which worked great in the previous version of FP.

Example url that worked before:

http://docs.google.com/spreadsheets/d/e/2PACX-1vQm6FWGVXEj7G3rCJFaeEx9SOqmAVKtoUs1NdIrgS62p881uhjyJ_rjdJTadUCe8HI8FNt3Ni9mJ5lO/pub?gid=778469003&single=true&output=tsv

 

I tried to revert back to FP 10.0, but it wouldn't install. Acrobat updated to DC 2019, so I wonder if FP 10.0 isn't compatible.

 

I've attached my collected test files in case you see something I missed.

 

I really appreciate your help.

collect-01-181025.zip

[Dan Korn]

Thanks for replying, Dan. I finally had a few minutes to test this out further. I've tried a couple of links on that page in additional to my own files, but I haven't been able to get the XDF URL to work. It works just fine if the file is local.

So to clarify, please answer these questions as accurately as you can:

1. If you add this a JavaScript rule with the syntax below to a sample job, like the Frodo Travel tutorial, what does it return when you click Validate in the Rule Editor? (It returns 5 for me.)
   

   var XDF = new ExternalDataFileEx("https://people.sc.fsu.edu/~jburkardt/data/csv/addresses.csv");
   return XDF.recordCount;

   
   

2. When it doesn't work, what relevant messages are there in the composition log (.msg) file? You will need to do a "Compose" (instead of a Preview) and click "View Log" to see the messages.
   

I also want to mention that most of my external files are Google sheets, which worked great in the previous version of FP.

Example url that worked before:

http://docs.google.com/spreadsheets/d/e/2PACX-1vQm6FWGVXEj7G3rCJFaeEx9SOqmAVKtoUs1NdIrgS62p881uhjyJ_rjdJTadUCe8HI8FNt3Ni9mJ5lO/pub?gid=778469003&single=true&output=tsv

This works fine for me in FP 10.1:

var XDF = new ExternalDataFileEx("http://docs.google.com/spreadsheets/d/e/2PACX-1vQm6FWGVXEj7G3rCJFaeEx9SOqmAVKtoUs1NdIrgS62p881uhjyJ_rjdJTadUCe8HI8FNt3Ni9mJ5lO/pub?gid=778469003&single=true&output=tsv");
return XDF.recordCount;

It returns 73.

I tried to revert back to FP 10.0, but it wouldn't install. Acrobat updated to DC 2019, so I wonder if FP 10.0 isn't compatible.

Acrobat DC 2019 requires FusionPro 10.1.

I've attached my collected test files in case you see something I missed.

Your test job works fine for me. The number 360 appears in the output.

I really appreciate your help.

No problem. But I'm still trying to clarify exactly what the problem is, more specifically than just that it "doesn't work." If you can answer my questions above, that will help me to help you.

[kjacobson]

1. If you add this a JavaScript rule with the syntax below to a sample job, like the Frodo Travel tutorial, what does it return when you click Validate in the Rule Editor? (It returns 5 for me.)

It returns 0

 

2. When it doesn't work, what relevant messages are there in the composition log (.msg) file? You will need to do a "Compose" (instead of a Preview) and click "View Log" to see the messages.

Error downloading URL: The requested security protocol is not supported.

Failed to open file

 

Avast Antivirus came up the first time I composed the file. I deactivated Avast, but the error still occurs.

[Dan Korn]

It returns 0

 

Error downloading URL: The requested security protocol is not supported.

Failed to open file

 

Avast Antivirus came up the first time I composed the file. I deactivated Avast, but the error still occurs.

Thank you for that information.

 

So we did change how FusionPro does downloading of files from the Internet on Windows, to support "https" URLs as well as "http". But what I don't understand is why that message about "The requested security protocol is not supported" would come up when requesting an "http" URL, because, unlike with "https", there should be no security protocol involved.

 

Also, because the server side (the endpoint from which you're downloading the file, such as docs.google.com) is the same for you and me, I'm trying to isolate whatever it is that's different on the client end, about either your machine and its settings, or some kind of setting in your Internet connection or gateway.

 

Some other questions for which the answers would be helpful:

1. What specific version of FusionPro 10.0 were you running before you upgraded?
   
2. Do you have another machine on which you could try installing FusionPro to see if it works there?
   
3. Can you try putting your machine on a different network? (Like, if it's a laptop, try it from home or from a different office.)
   
4. Does the URL open up in a web browser for you?
   
   • From your default browser? (What browser is that?)
     
   • From Internet Explorer?
     

Edited October 31, 2018 by Dan Korn

[kjacobson]

1. What specific version of FusionPro 10.0 were you running before you upgraded?
   FusionPro 10.0.26
   
2. Do you have another machine on which you could try installing FusionPro to see if it works there?
   I will try this tonight.
   
3. Can you try putting your machine on a different network? (Like, if it's a laptop, try it from home or from a different office.)
   I have tried from home, work, and a couple of my neighbors. Same results.
   
4. Does the URL open up in a web browser for you?
   
   • From your default browser? (What browser is that?)
     
   • From Internet Explorer?
     Firefox is my default browser. The URL opens in every browser (Firefox, IE, Chrome)
     

[kjacobson]

The problem has been fixed!

I will admit, that I am not usually very eager to update most software. I usually wait until I absolutely have to update, this includes Windows updates. I just installed 6 updates for Windows and now the problem is fixed.

 

Thank you Dan for working with me on this. Your questions directed me to delve deeper into my own system. Hopefully this will also help others if they experience the same issue.