Skip to content

(re)add read-spotgins (v2 & v3) + add read-rtklib #186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
PierreS-alpha wants to merge 3 commits into
base: dev
Choose a base branch
from
Open

(re)add read-spotgins (v2 & v3) + add read-rtklib #186

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
686429d
(re)add read-spotgins (v2 & v3)
PierreS-alpha Jun 11, 2026
46a4aba
add read_rtklib
PierreS-alpha Jun 11, 2026
a01fbe7
add rtklib format support to rawformats.conf and update readfmtdata_g…
PierreS-alpha Jun 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CODE/etc/rawformats.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ winston|EARTHWORM|EarthWorm Winston Wave Server data request||host:port
gipsy|GNSS|JPL GIPSY-OASIS .tdp file|ANTENNA,RECEIVER,XYZ|fullpath of root directory containing YYYY/FID/*.tdp files
gipsyx|GNSS|JPL GipsyX .tdp file|ANTENNA,RECEIVER,XYZ|fullpath of root directory containing YYYY/FID/*.tdp files
globkval|GNSS|MIT GAMIT/GLOBK VAL file||fullpath of directory containing the .VAL file
spotgins-enu-v2|GNSS|SPOTGINS solutions - version 2 (<2025-08)||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
spotgins-ippp|GNSS|SPOTGINS IPPP time series||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
spotgins-enu|GNSS|SPOTGINS ENU solutions - v2&3||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
spotgins-ippp|GNSS|SPOTGINS IPPP time series (ITES)||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
gamit-pos|GNSS|MIT GAMIT/GLOBL POS time series||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
pbogps-pos|GNSS|PBO GPS POS time series||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
usgs-rneu|GNSS|USGS RNEU text file||fullpath of file(s) with bash wildcard facilities
Expand All @@ -43,3 +43,4 @@ sbe37-ascii|ASCII|sbe37 OBP text file|DATA_ERROR,DATA_DECIMATE|fullpath of file(
mc3|WO|MainCourante database||MC3 name
mat-file|WO|Matlab MAT-file|T,D,E|fullpath of .mat file
genform|WO|GENFORM database||FORM name|
rtklib|GNSS|RTKLIB solution file||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
2 changes: 1 addition & 1 deletion CODE/matlab/readfmtdata.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@
case {'miniseed','seedlink','arclink','combined','fdsnws-dataselect'}
D(n) = readfmtdata_miniseed(WO,P,N(n),F);

case {'globkval','gipsy','gipsyx','gipsy-tdp','usgs-rneu','ies-neu','ogc-neu','ingv-gps','sbe37-ascii','spotgins-enu-v2','spotgins-ippp','gamit-pos','pbogps-pos'}
case {'globkval','gipsy','gipsyx','gipsy-tdp','usgs-rneu','ies-neu','ogc-neu','ingv-gps','sbe37-ascii','spotgins-enu','spotgins-ippp','gamit-pos','pbogps-pos','rtklib'}
D(n) = readfmtdata_gnss(WO,P,N(n),F);

case {'hyp71sum2k','fdsnws-event','scevtlog-xml'}
Expand Down
194 changes: 178 additions & 16 deletions CODE/matlab/readfmtdata_gnss.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,13 @@
% data format: jjjjj.jj E N V dE dN dV yyyymmddhhmmss yyyy.yyyyyyyyy
% node calibration: no .CLB file or 4 components (East, North, Up) in meters and (Orbit)
%
% format 'spotgins-enu'
% type: SPOTGINS solutions - supports v2 and v3 formats (auto-detection)
% filename/url: P.RAWDATA (use $FID to point the right file/url)
% data format v2: jjjjj.jj E N V dE dN dV yyyymmddhhmmss yyyy.yyyyyyyyy
% data format v3: MJD DispEast DispNorth DispUp SigmaEast SigmaNorth SigmaUp CorrEN CorrEU CorrNU yyyy-mm-ddTHH:MM:SS DecimalYear Const Flag DateOfExe GinsVersion PrairieVersion
% node calibration: no .CLB file or 4 components (East, North, Up) in meters and (Orbit)READFMTDATA_GNSS subfunction of readfmtdata.m
%
% format 'spotgins-ippp'
% type: GINS IPPP solutions
% filename/url: P.RAWDATA (use $FID to point the right file/url)
Expand Down Expand Up @@ -76,6 +83,16 @@
% data format: ascii
% node calibration: no .CLB file or 4 components (East, North, Up) in meters and (Orbit)
%
% format 'rtklib'
% type: RTKLIB solution file (ENU baseline, FLH absolute, or XYZ ECEF)
% filename/url: P.RAWDATA (use $FID to point the right file/url)
% data format: yyyy/mm/dd HH:MM:SS.sss a b c Q ns s_a s_b s_c sdAB sdBC sdAC age ratio
% header lines start with %%; coordinate type auto-detected from header keywords
% ENU: e-baseline(m) n-baseline(m) u-baseline(m)
% FLH: latitude(deg) longitude(deg) height(m) - converted to UTM
% XYZ: x-ecef(m) y-ecef(m) z-ecef(m) - converted to UTM
% node calibration: no .CLB file or 4 components (East, North, Up) in meters and (Orbit)
%
%
% Authors: François Beauducel and Jean-Bernard de Chabalier, WEBOBS/IPGP
% Created: 2016-07-10, in Yogyakarta (Indonesia)
Expand Down Expand Up @@ -306,19 +323,66 @@
end
%D.ITRF_YEAR = 'ITRF08';


% -----------------------------------------------------------------------------
case 'spotgins-enu-v2'
% format exemple
%#jjjjj.jjjjjjjj _____E _____N _____U ____dE ____dN ____dU yyyymmddHHMMSS yyyy.yyyyyyy Const Dateofexe GinsVersion
% 52670.83876160 0.055822 0.051638 0.005578 0.001263 0.001163 0.004899 20030131200749 2003.0844898 G 250404_185253 VALIDE_24_2
% 52671.50195600 0.057207 0.054240 -0.004722 0.000705 0.000619 0.002600 20030201120249 2003.0863067 G 250404_185253 VALIDE_24_2

case 'spotgins-enu'
% Supports both v2 and v3 SPOTGINS formats
%
% v2 format example:
%#jjjjj.jjjjjjjj _____E _____N _____U ____dE ____dN ____dU yyyymmddHHMMSS yyyy.yyyyyyy Const Dateofexe GinsVersion
% 52670.83876160 0.055822 0.051638 0.005578 0.001263 0.001163 0.004899 20030131200749 2003.0844898 G 250404_185253 VALIDE_24_2
% 52671.50195600 0.057207 0.054240 -0.004722 0.000705 0.000619 0.002600 20030201120249 2003.0863067 G 250404_185253 VALIDE_24_2
%
% v3 format example:
%#MJD DispEast DispNorth DispUp SigmaEast SigmaNorth SigmaUp CorrEN CorrEU CorrNU yyyy-mm-ddTHH:MM:SS DecimalYear Const Flag DateOfExe GinsVersion PrairieVersion
% 51668.5 0.080276 -2.058358 0.009854 0.000312 0.000439 0.001339 -0.028963 -0.085824 -0.011884 2000-05-04T12:00:00 2000.340164 G 0 250910_120313 25_1 v56
% 51669.5 0.077154 -2.057207 0.005923 0.000257 0.000371 0.001109 -0.017412 -0.060937 -0.124317 2000-05-05T12:00:00 2000.342896 G 0 250910_120313 25_1 v56

fdat = sprintf('%s/%s.dat',F.ptmp,N.ID);
wosystem(sprintf('rm -f %s',fdat),P);

% Detect format version from first file
format_version = '';
if ~isempty(F.raw)
first_raw = F.raw{1};
if strncmpi('http',first_raw,4)
[s,header] = wosystem(sprintf('curl -s -S "%s" | head -20',first_raw),P);
else
[s,header] = wosystem(sprintf('head -20 %s',first_raw),P);
end
if s == 0
if ~isempty(strfind(header,'SPOTGINS SOLUTION [POSITION] v3')) || ~isempty(strfind(header,'MJD DispEast'))
format_version = 'v3';
elseif ~isempty(strfind(header,'SPOTGINS SOLUTION [POSITION] v2')) || ~isempty(strfind(header,'jjjjj.jjjjjjjj'))
format_version = 'v2';
else
% Try to detect from data columns (fallback method)
if strncmpi('http',first_raw,4)
[s,sample] = wosystem(sprintf('curl -s -S "%s" | grep -v "^#" | head -1',first_raw),P);
else
[s,sample] = wosystem(sprintf('grep -v "^#" %s | head -1',first_raw),P);
end
if s == 0 && ~isempty(sample)
cols = length(strsplit(strtrim(sample)));
if cols >= 17 % v3 has 17+ columns
format_version = 'v3';
elseif cols >= 11 % v2 has 11+ columns
format_version = 'v2';
end
end
end
end
end

if isempty(format_version)
format_version = 'v2'; % default fallback
fprintf('%s: ** INFO ** Could not detect SPOTGINS format version, assuming v2.\n',wofun);
else
fprintf('%s: ** INFO ** Detected SPOTGINS format %s.\n',wofun,format_version);
end

for a = 1:length(F.raw)
fraw = F.raw{a};
cmd0 = sprintf('awk ''/^[^#]/ {print}'' >> %s',fdat); % removes header lines
cmd0 = sprintf('awk ''/^[^#]/ {print}'' >> %s',fdat); % removes header lines
if strncmpi('http',fraw,4)
s = wosystem(sprintf('curl -s -S "%s" | %s',fraw,cmd0),P);
if s ~= 0
Expand All @@ -331,20 +395,31 @@
fprintf('%s: ** WARNING ** Raw data "%s" not found.\n',wofun,fraw);
end
end


% load the file

% load the file
if exist(fdat,'file')
dd = dlmread(fdat);
else
dd = [];
end

if ~isempty(dd)
t = dd(:,1) + 678941.5007; % converts MJD to datenum
d = [dd(:,2:4),zeros(size(dd,1),1)]; % North(mm),East(mm),Up(mm) => E(m),N(m),U(m),Orbit
e = dd(:,5:7);
switch format_version
case 'v3'
% v3 format: MJD DispEast DispNorth DispUp SigmaEast SigmaNorth SigmaUp ...
t = dd(:,1) + 678941.5007; % converts MJD to datenum
d = [dd(:,2:4),zeros(size(dd,1),1)]; % DispEast,DispNorth,DispUp,Orbit => E(m),N(m),U(m),Orbit
e = dd(:,5:7); % SigmaEast,SigmaNorth,SigmaUp

case 'v2'
% v2 format: jjjjj.jj E N U dE dN dU ...
t = dd(:,1) + 678941.5007; % converts MJD to datenum
d = [dd(:,2:4),zeros(size(dd,1),1)]; % E,N,U,Orbit => E(m),N(m),U(m),Orbit
e = dd(:,5:7); % dE,dN,dU
end

e(e<min_error) = min_error;
fprintf('%d data imported.\n',size(dd,1));
fprintf('%d data imported (format %s).\n',size(dd,1),format_version);
else
fprintf('no data found!\n')
t = [];
Expand All @@ -355,7 +430,7 @@

% -----------------------------------------------------------------------------
case 'spotgins-ippp'
% From J.S. - ITES Strasbourg
% From J.S. - ITES Strasbourg
% format example
% !yyyymmdd hhmmss yyyy.yyyyyyyyy jjjjj.jj X_position Y_position Z_position dX dY dZ E N V dE dN dV
% 20160723 65619 2016.558521561 57592.29 4182067.152057 570976.439258 4765940.539811 0.000611 0.000218 0.000673 -0.006574 -0.008848 -0.014844 0.000205 0.000307 0.000859
Expand Down Expand Up @@ -576,6 +651,93 @@
e = [];
end

% -----------------------------------------------------------------------------
case 'rtklib'
% RTKLIB solution file
% Header lines start with %. Coordinate type auto-detected from header keywords:
% 'e-baseline' => ENU, 'x-ecef' => XYZ, 'latitude' => FLH (default)
% Data format: yyyy/mm/dd HH:MM:SS.sss a b c Q ns s_a s_b s_c sdAB sdBC sdAC age ratio

fdat = sprintf('%s/%s.dat',F.ptmp,N.ID);
wosystem(sprintf('rm -f %s',fdat),P);

% detect coordinate type from file header
initype = 'FLH'; % RTKLIB default output is FLH
if ~isempty(F.raw)
fraw = F.raw{1};
if strncmpi('http',fraw,4)
[~,hdr] = wosystem(sprintf('curl -s -S "%s" | head -30',fraw),P);
else
[~,hdr] = wosystem(sprintf('head -30 "%s"',fraw),P);
end
% Be more robust: check multiple keyword variants
if ~isempty(strfind(lower(hdr),'x-ecef')) || ~isempty(strfind(lower(hdr),'y-ecef')) || ~isempty(strfind(lower(hdr),'z-ecef'))
initype = 'XYZ';
elseif ~isempty(strfind(lower(hdr),'e-baseline')) || ~isempty(strfind(lower(hdr),'n-baseline'))
initype = 'ENU';
elseif ~isempty(strfind(lower(hdr),'latitude')) || ~isempty(strfind(lower(hdr),'longitude'))
initype = 'FLH';
end
end
fprintf('%s: ** INFO ** RTKLIB coordinate type: %s\n',wofun,initype);

% extract data: convert 'yyyy/mm/dd HH:MM:SS.sss' date/time to 6 numeric columns
% resulting columns: y m d H M S a b c Q ns s_a s_b s_c sdAB sdBC sdAC age ratio
for a = 1:length(F.raw)
fraw = F.raw{a};
cmd0 = sprintf(['awk ''/^[^%%]/{split($1,d,"/");split($2,t,":");' ...
'print d[1],d[2],d[3],t[1],t[2],t[3],$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15}'' >> %s'],fdat);
if strncmpi('http',fraw,4)
s = wosystem(sprintf('curl -s -S "%s" | %s',fraw,cmd0),P);
if s ~= 0
break;
end
else
s = wosystem(sprintf('cat "%s" | %s',fraw,cmd0),P);
end
if s ~= 0
fprintf('%s: ** WARNING ** Raw data "%s" not found.\n',wofun,fraw);
end
end

% load the file
% columns: 1=year 2=month 3=day 4=hour 5=min 6=sec 7=a 8=b 9=c
% 10=Q 11=ns 12=s_a 13=s_b 14=s_c 15=sdAB 16=sdBC 17=sdAC 18=age 19=ratio
if exist(fdat,'file')
dd = load(fdat);
else
dd = [];
end
if ~isempty(dd)
t = datenum(dd(:,1),dd(:,2),dd(:,3),dd(:,4),dd(:,5),dd(:,6));
switch initype
case 'ENU'
% e-baseline(m), n-baseline(m), u-baseline(m)
% sigma columns: s_a=sde, s_b=sdn, s_c=sdu
d = [dd(:,7),dd(:,8),dd(:,9),zeros(size(dd,1),1)]; % E,N,U,Orbit
e = [dd(:,12),dd(:,13),dd(:,14)]; % sde,sdn,sdu
case 'FLH'
% latitude(deg), longitude(deg), height(m) - convert to UTM
% sigma columns: s_a=sdn, s_b=sde, s_c=sdu (N before E in RTKLIB FLH output)
[E_utm,N_utm] = ll2utm(dd(:,7),dd(:,8));
d = [E_utm(:),N_utm(:),dd(:,9),zeros(size(dd,1),1)]; % E_UTM,N_UTM,h,Orbit
e = [dd(:,13),dd(:,12),dd(:,14)]; % sde(=s_b),sdn(=s_a),sdu
case 'XYZ'
% x-ecef(m), y-ecef(m), z-ecef(m) - convert to UTM
% sigma columns: s_a=sdx, s_b=sdy, s_c=sdz
[enu_xyz,e_xyz] = cart2utm(dd(:,7:9),dd(:,12:14));
d = [enu_xyz,zeros(size(dd,1),1)]; % E,N,U,Orbit
e = e_xyz;
end
e(e<min_error) = min_error;
fprintf('%d data imported (type: %s).\n',size(dd,1),initype);
else
fprintf('no data found!\n')
t = [];
d = [];
e = [];
end

% -----------------------------------------------------------------------------
case 'sbe37-ascii'
% format example
Expand Down