Path names problem in 1.5 IDE's platforms.txt

[tchilton]

Although it seems to work with avrdude, there are consistency problems in the platforms.txt file's handling of paths. All the paths in tools.avrdude have Linux forward slashes, which results in the path names having a mix of forward and backslashes on the Windows platform. This can be seen by turning on verbose mode for an upload or opening the platforms.txt

This will be a problem for other integrating other programmers as they will probably copy the avrdude section (like I did)

Suggest that the platforms.txt should have both .windows and .Linux constructs by default, or possibly the IDE performs some form of sanity check on pathnames before attempting to execute them - although differentiating paths and switches could be tricky !

[matthijskooijman]

Is having a mix an actual problem on Windows? I think we would have heard complaints if it were? Or is avrude internally fixing this up and are there other tools that don't do this?

[tchilton]

Yes, Windows only accepts backslash and Linux only forward slash. Assume AVRDUDE must map these since its an argument passed, but I found the problem when trying to do the remote SCP/SSH programmer and hit the problem, hence why I raise it.

There is no point having some confusion built in, particularly if the Arduino platform can do .windows and .Linux suffixes for each OS. Failure to fix it means that others will have to :

1. identify that the problem exists
2. work-around it in their code
3. Platform independence will not be maintained during that period.

So, why not just fix it in the platforms.txt file ???

For an example, look at this line from platforms.txt - the /U flag.
tools.avrdude.upload.pattern="{cmd.path}" "-C{config.path}" {upload.verbose} -p{build.mcu} -c{upload.protocol} -P{serial.port} -b{upload.speed} -D "-Uflash:w:{build.path}/{build.project_name}.hex:i"

The result is a path with the build path being OS generated and backslash on windows, followed by a forward slash and the project name.

[matthijskooijman]

Right, I understand the problem now.

Fixing this by duplicating every command that uses a path with a windows-specific version is possible, but I don't like that much (because of the duplication). If we can thinking of a more generic way to fix this, that would be better.

However, as you already pointed out, we can't just globally replace / with \ in the command, since that will screw up commandline switches. I can't think of any other elegant way to fix this right now, so duplicating the commands might be the way to go.

Disclaimer: I'm not not an official dev, so my opinion isn't binding in any way :-)

[jantje]

Can you be more specific where it gives problems? I mean: I assume that with SCP/SSH you mean the yun upload. I have implemented this in eclipse and did not have any problems.
I also implemented the teensy upload using the boards.txt also without problems (on the / versus \ side)

[tchilton]

Hi Jantje.
This is nothing to do with Yun. I am implementing a remote upload capability - IDE to an Arduino that is connected to via a Raspberry Pi at the far end of a wireless link. This is when I found the problem. My mods and scripts will be released shortly - SCP and an SSH.

My point is that the directory rules for each OS are well documented and the IDE needs to follow them, like other standards, otherwise when people try and do any form of extension in this area, they will have to find this issue and work around it.

So, for consistency the IDE's configuration files need to handle this correctly for each OS. Yes, AVRDUDE does have some work-around in place for this, but we should not rely on this or assume that all other tools will have the same capabilities to work around this bug.

[cmaglie]

@tchilton
Windows is able to handle both forward and back slashes (and also a mix of them), I noticed that a while ago, but a quick search on google confirms it:

• http://bytes.com/topic/python/answers/23123-when-did-windows-start-accepting-forward-slash-path-separator
• http://qt-project.org/forums/viewthread/692

Probably it's better if you can show your specific problem?

[amulya349]

We can use a keyword like &slash& (or anything) which we will use in place of '/' and avrdude will itself replace that keyword with the appropriate form of slash for the respective platform.

[cmaglie]

Yeah, a property named {slash} or {file.sep} to be used in configuration files (that is set to / or \, depending on the OS) is probably the best solution for this issue since avoids the need to duplicate every configuration.

[jantje]

 Yeah, a property named {slash} or {file.sep} to be used in configuration files (that is set to / or \, depending on the OS) is probably the best solution for this issue since avoids the need to duplicate every configuration.

In my humble opinion this will make the the platform.txt file less readable.
I still don't understand what the problem is that should be fixed. As such I don't understand the benefit to add complexity.
I agree that a multiplatform environment (compile on windows-> upload on mac/linux/windows) adds complexity. However the current platform.txt data does that in my case.
IMHO the best solution is that the IDE does not know the uploading OS. As far as I understand this is the current situation.
I mean: Are you willing to implement a target platform detection method? Isn't the yun way the way to go?

@tchilton maybe you should take a look at how yun does the upload. Basically it opens a ssh shell and runs some commands (without parameters). If you implement these commands on your target OS -and have them in your path- you can upload as if you have a yun. I did so with a linksys router and openwrt (before linino became openwrt)
This means no messing around with boards.txt and not cross-platform path separator issues.

[cmaglie]

Hi @jantje
maybe I've expressed myself wrong, as I already pointed out, windows is perfectly capable of handling forward slashes (or even a mix of forward and backward slashes on the same command line), so I basically agree with you: I'd like to see the actual problem @tchilton is facing before going on with any solution on that.

Just for the record: the IDE already does OS detection with the Base.isWindows() method here
https://github.com/arduino/Arduino/blob/master/app/src/processing/app/Base.java#L1414

C

[jantje]

@cmaglie
I know the IDE is doing OS detection. I am talking about "target platform detection method?" like in "target platform= platform running avr-dude". In other words when the upload is done running a remote shell (like for yun upload via wifi): do you want to bother finding out what os is running the remote shell service to modify the commands the IDE is launching?

[cmaglie]

Ah ok, I missed that "target platform" part.

Any target platform, especially if it's a remote target, is on its own to handle platform specific stuff, no remote OS detection performed by the IDE.

The solution I proposed was to provide a {sep} variable for the recipes, but only for the OS that is currently running the IDE, so for example the recipe:

... -Uflash:w:{build.path}/{build.project_name}.hex:i ...

can be rewritten as:

... -Uflash:w:{build.path}{sep}{build.project_name}.hex:i ...

but, again, before proceeding I'll wait to see an example that shows the actual problem.

[tchilton]

Lots of comments.. Lots of questions

1. Windows handling forward slash - didn't know that, yes It works on Windows 7. However the coverage of it is limited - even within Windows. Try for example going to the command line and doing a cd c:\pro - and completion gives you c:\Program Files (x86) as an example. Try the same with forward slash - nothing. Also applications that parse parameters etc. may not expect it, so you can't rely on it until this is far better known and supported. In my specific case, I was doing commands with batch files, Visual C++ 2010 and PuTTY / WinSCP. Forget which one had the problem but this is how I found it as something didn't work as expected, I fixed the slash in the config file and it started working. Its also true that the same the other way around on Linux would not work as its an escape. Presumably the reason the config files have parameter=, parameter.windows= and parameter.linux= is to allow for such differences. My personal view here is that people are over-analysing this. Use the existing functionality for what someone presumably developed it for.
2. Re the {slash} option, that would be confusing - does it mean a path separator or a switch ?. I agree that whatever the {macro} is, it will be more confusing than using the above
3. I'm not asking for any target platform detection, I'm asking for correct handling of the local OS platform. If someone chooses extend the platform for whatever their need, it should be obvious how to by looking at the existing examples in the board.txt / platforms.txt / programmers.txt files. Keep the learning curve as low as possible - isn't this what Arduino is trying to do ??
4. Re my approach and how Yun does it. my configuration is a Windows PC to a Raspberry Pi and onwards to a Mega 2560 over SPI. I don't have a Yun, The Pi does not have interfaces like a Yun and I have no interest in developing something bespoke to emulate a product I don't have. I'm talking to a Linux box, you talk to it via SSH and SCP which are far better defined and understood standards. I chose to do it this way because its logical to me and probably millions of others who understand Linux. My scripts work well (for my requirement) and give me plenty of control since they are not compiled up, I just need one entry added to each of the 3 configuration files (above) to make this work. I can now sit in the IDE on a PC, edit and compile, hit the upload button and it copies the file across to my Pi over wireless, then SSH's into the Pi and runs a local script that invokes the local programmer with my specific configuration options I need for that platform and my Arduino gets an firmware upload. I also get all the console output back in the IDE, so its easy to see what's happening. All clean and easy to understand if someone needs to lift the lid on the scripts to debug anything. Any references about AVRDude's handling things are null since I'm not using AVRDude on the host PC, I invoke it with SOME of the parameters and some extra local parameters on the remote host. The only issue I have now relates to noise on the SPI interface, but that's not in the scope of this forum and I know how to fix it, I've just not had the time to. I will release the scripts In the near future for anyone else who wants to use them. I know from conversations with others that they also want a similar approach. See Add new programmer class - for remote upload #2047

I raised this defect since its caused problems (as it did in my case). Windows common knowledge is backslash is the path separator, so why make it harder than this in the example config files ??

I've fixed my issue, I've reported the problem I found in this thread. Take it or leave it - I don't care any more.

[bobc]

Whether the type of slash in a parameter depends entirely on user application. Some may handle a mix, some may not. However the correct method on Windows is to use backslash, therefore this is what Arduino should pass to external tools.

[lmihalkovic]

Isn't it amazing that so much can be written about such a trivial issue with so little actual result!? This issue is not unlike the normalization step that exist in IP to ensure a consistent endian-ness accross the board. The point is that having to deal with normalized versus specific representation of the same information is something programmers have had to deal with for at least 40years.

[cmaglie]

Isn't it amazing that so much can be written about such a trivial issue with so little actual result!?

@lmihalkovic, at least read the issue before doing such harsh comments.

This issue started from the false assumption that Windows only handle paths with backslash while is perfectly capable of handling path with both forward and backslash (even mixed). The OP failed to show a use case when the actual setup fails, so I'm closing this issue as "works for me".

[lmihalkovic]

This issue started from the false assumption that Windows only handle paths with backslash while is perfectly capable of handling path with both forward and backslash (even mixed). The OP failed to show a use case when the actual setup fails, so I'm closing this issue as "works for me".

@cmaglie Precisely! I was referring to the amount of discussion, proposal, opinions that got expressed around something as small as a path 'problem'. It would surely be nice if the code/configs were more consistent when dealing with paths, but it can be hapily ignored due to the foregiveness of Windows.

IMHO there are many more entries showing a similar pattern of long discussions with elaborate arguments on various sides of mis-diagnosed 'problems', which prompt me to regularly say that your codebase is full of opportunities for useful improvements when your peel-off the layers of misunderstanding around these issues/features. Or in the words of someone much more eloquent: it is not that they cannot see the answer, it is that they cannot see the question